[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: how to open very large txt files (>6G)

From   Neil Shephard <[email protected]>
To   [email protected]
Subject   Re: st: how to open very large txt files (>6G)
Date   Tue, 30 Sep 2008 13:15:15 +0100

Ada Ma wrote:
> Dear Statalisters,
> I need to check if the data looks OK - the data should be delimited by
> pipe (|) so I want to check if there is any double quote in the data
> before I read it into Stata.  The data is in txt format.  I wonder
> which are text editors would allow me to open that up.  All
> suggestions are welcomed.

This will be _highly_ dependent on your Operating System which you've
not mentioned.

Under M$-Windows (XP) there is a limit of 2Gb on the allocation of
contiguous memory to any one application and a 4Gb limit on that OS in
total for 32-bit systems.  These are higher for 64-bit systems, but it
depends on the version of the OS you have (see

Its perhaps less of a problem under *NIX systems, but you'll need the
hardware and to have it configured to support allocation of >4Gb.

Personally if I wanted to check for double quotes in a file I'd use
'grep '"' [filename]', to search for a regular expression  (grep is a
standard command on *NIX-like systems).  Thanks to Nick Cox I was made
aware of the analogous function under MS-DOS which is the 'find'
command.  I'm not sure about how to actually search for a double quote
though as the string you want to search for has to be encapsulated in
double quotes and a quick test of escaping it ("\"") didn't work.  find
/? will give you more information and google may help to point you in
the right direction.


This  message  may  contain  confidential and  privileged  information.
If you  are not the  intended recipient  you should not  disclose, copy
or distribute information in this e-mail or take any action in reliance
on its contents.  To do so is strictly  prohibited and may be unlawful.
Please  inform  the  sender that  this  message has  gone astray before
deleting it.  Thank you.

2008 marks the 60th anniversary of the NHS.  It's an opportunity to pay
tribute to the NHS staff and volunteers who help shape the service, and
celebrate their achievements.

If you work for the NHS  and  would like  an NHSmail  email account, go

*   For searches and help try:

© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index