Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Problems with copying a file from the Internet.


From   Christopher Baum <kit.baum@bc.edu>
To   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   Re: st: Problems with copying a file from the Internet.
Date   Mon, 12 Sep 2011 15:52:53 -0400

<>	
Guo replied to Ulrich:


I assume the server throws a HTTP 302 redirection when it cannot
identify your request as made from a browser (using HTTP_USER_AGENT).
This is commonly done to avoid scripts scraping data automatically.

The ideal way to solve this would be to switch the HTTP_USER_AGENT
identifier to a standard browser (e.g. IE, Mozilla), but I'm not sure
how to do this.

Guo

On 12 September 2011 11:07, Ulrich Kohler <kohler@wzb.eu> wrote:
> hi all,
>
> A program of mine wants to copy this file
>
>
 http://www.iso.org/iso/list-en1-semic-3.txt

>
> from the internet using something like
>
> . copy
 http://www.iso.org/iso/list-en1-semic-3.txt
 foo.txt
>
> With Stata 12 I get the error
>
> --
> file
 http://www.iso.org/iso/list-en1-semic-3.txt
 not found
> server says file temporarily redirected to
>
 http://www.iso.org/iso/home.html

> r(601);
> --
>
> although the file is shown without problems in a browser.
>
> Does anybody has an explanation for this?
>
> Uli
>

I think this is probably what is going wrong. Verified problem, in both Stata 12 and Stata 11.2. Can't even open the file with fopen() in Mata. Probably the easiest workaround (as I do not think you can tinker with Stata's HTTP_USER_AGENT setting) is to use some local utility, such as curl or wget, e.g. within 
a do- or ado-file,

tempname ctylist
!curl  http://www.iso.org/iso/list-en1-semic-3.txt > `ctylist'
type `ctylist'

works on my *nix system; wget may be the utility to use on others.

Kit

Kit Baum   |   Boston College Economics & DIW Berlin   |   http://ideas.repec.org/e/pba1.html
                              An Introduction to Stata Programming  |   http://www.stata-press.com/books/isp.html
   An Introduction to Modern Econometrics Using Stata  |   http://www.stata-press.com/books/imeus.html




*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index