Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: collecting raw data from the web via browser automation


From   "Austin Nichols" <austinnichols@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   st: collecting raw data from the web via browser automation
Date   Mon, 22 May 2006 17:21:45 -0400

Dear _all,

I am dipping my toe into the cold pool of meta-analysis, and have been
really happy with the results Google Scholar generates, producing
citations for which electronic or physical copies can then be hunted
down, but I don't want to enter bibliographic data on 600 or 900 or
more studies by hand.  Google Scholar has a nice way to set
Preferences so that links to bibliographic info are generated in the
search results, but I don't use BibTeX or EndNote or any of those
things--I use Stata, and I want to automate the whole process of
seaching and saving those data (which look like
http://scholar.google.com/scholar.bib?q=info:nmXVGJVxYjQJ:scholar.google.com/&output=citation
by the way) and infiling them into Stata so I can have a nice database
of articles made for me on any set of search terms I put in.

The trouble is this: the link to bibliographic data is not a static
page; it is generated on the fly, so Stata cannot -copy- to a local
file to -infile- the info.  I will need a browser to browse to that
location, and then save the results.  Does anyone have a freeware
solution to this problem?  I have access to several varieties of
Windows and Unix/Linux, but no Mac OS options.  What I am thinking is
that if there is a command line browser with the option to save the
page to disk, I can just invoke the page and save it with a single
line of code that begins with the -shell- command, and then infile it
with another that begins -infile-.

Yours hopefully,
Austin

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index