Statalist The Stata Listserver

[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: collecting raw data from the web via browser automation

From   "Austin Nichols" <>
Subject   st: collecting raw data from the web via browser automation
Date   Mon, 22 May 2006 17:21:45 -0400

Dear _all,

I am dipping my toe into the cold pool of meta-analysis, and have been
really happy with the results Google Scholar generates, producing
citations for which electronic or physical copies can then be hunted
down, but I don't want to enter bibliographic data on 600 or 900 or
more studies by hand.  Google Scholar has a nice way to set
Preferences so that links to bibliographic info are generated in the
search results, but I don't use BibTeX or EndNote or any of those
things--I use Stata, and I want to automate the whole process of
seaching and saving those data (which look like
by the way) and infiling them into Stata so I can have a nice database
of articles made for me on any set of search terms I put in.

The trouble is this: the link to bibliographic data is not a static
page; it is generated on the fly, so Stata cannot -copy- to a local
file to -infile- the info.  I will need a browser to browse to that
location, and then save the results.  Does anyone have a freeware
solution to this problem?  I have access to several varieties of
Windows and Unix/Linux, but no Mac OS options.  What I am thinking is
that if there is a command line browser with the option to save the
page to disk, I can just invoke the page and save it with a single
line of code that begins with the -shell- command, and then infile it
with another that begins -infile-.

Yours hopefully,

*   For searches and help try:

© Copyright 1996–2023 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index