Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: AW: Reclaiming RAM. . .


From   Joachim Landström <joachim.landstrom@fek.uu.se>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: AW: Reclaiming RAM. . .
Date   Tue, 10 Nov 2009 09:53:20 +0100

us_actg_data is just approx 88 MB but the big one is us_data_ret. It claims 368 MB. I guess I may come down from 1000M in memory allocation but have been experiencing problems when working on us_data_ret having less then 1000M.

It might be that when I start Stata it does not use all 1000M since it is only idle. I'll try to bring down the memory to 100M after the session is over to see what happens.

/Joachim



Quoting Martin Weiss <martin.weiss1@gmx.de>:


<>

" I have added the commands "clear all" and discard without
significant effect."


-clear all- implies -discard- so you can economize on typing... How big is
"us_actg_data"? With three nested loops and -joinby- action, searches for
-duplicates- and so on, it it conceivable that your computer is stressed.
The -set mem 1000M- continues to affect it even after -clear all-, so you
might also want to -set mem 100M- afterwards to see whether that has any
effect on the memory usage...


HTH
Martin


-----Ursprüngliche Nachricht-----
Von: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von Joachim
Landström
Gesendet: Dienstag, 10. November 2009 08:47
An: statalist@hsphsun2.harvard.edu
Betreff: st: Reclaiming RAM. . .

Hi I am having a minor problem when I run Stata pertaining to how it
uses RAM that I wish to query about. This would not have been a
problem had I not been working with rather large tables and so end up
with little free RAM.

As a background note: I use an up to date Windows Vista Ultimate,
Stata/MP 10.1 on a Intel Core2 processor having 2 GB RAM and automatic
page file size.

The profile.do invokes the following commands:
set maxvar 2048
set matsize 100
set memory 1000M
set virtual off

When I start up Stata the computer uses 32 percent of available RAM.

I then run a session of code where I essentially loads different
tables, modifies them and merges them. I do not use any non-native
ados. (See code below). When i run this session the computer uses up
to 95-100 percent RAM. Note that the code is still in development and
so I restrict loops and so on to just a few years.

After the session is finished the computer still uses 80 percent and
the computer feels rather unresponsive. To come to gripps with the
problem I have added the commands "clear all" and discard without
significant effect.

If I choose to close and restart Stata I again come back to the
original state where it just uses 32 percent RAM.

Does somebody have an inkling about what goes on and can somebody
suggest a remedy. . .

/Joachim

And yes, here is the code:
************ Beginning of code ********************

local beginTime = c(current_time)
display in yellow "Program start at `beginTime' on $S_DATE"

clear

use us_actg_data

quietly: summarize year
local minYear = 1995 + 1 // + 1 since first year BS data is used for sorting
local maxYear = 1997

local i = `minYear'

preserve

local size = "S"
local bmGroups = "H L M"

foreach x of local size ///
{
	foreach y of local bmGroups ///
	{
		while `i' <= `maxYear' ///
		{
			tempfile `x'`y'`i'id `x'`y'`i'data

			display in yellow "Selection based on `x' and `y'"
			drop if year != `i'
			drop if SMB != "`x'"
			drop if HML != "`y'"

			quietly summarize WC08001
			return list
			generate double totalMC = r(sum)
			generate double weightMC = WC08001 / r(sum)

			drop WC* BM PE totalMC

			// save ``x'`y'`i'id'  not really necessary

			joinby dscd using us_data_ret

			rename year actgYear
			label variable actgYear "Year for sort"

			generate int riYear = yofd(date)
			format riYear %ty
			label variable riYear "Year for Total Return Index"

			drop if riYear != `i' + 1 // Market data AFTER
sorting on size and bmGroups

			label data "Total Return for sort on `x' & `y' in
`i'"

			generate double portfolio`x'`y'totalreturn =
totalReturn * weightMC

			sort date
			by date: egen double tpr = total(
portfolio`x'`y'totalreturn )
			label variable tpr "Total Portfolio Return"

			drop dscd actgYear SMB HML weightMC id ///
			riYear totalReturn portfolio`x'`y'totalreturn

			duplicates drop

			tsset date, daily delta(7)

			save Temp`x'`y'`i'data, replace

			display in yellow "Done with `i' for `x' & `y' at
$S_TIME, $S_DATE."

			if `i' == `maxYear' ///
			{
				display "Time to merge portfolio returns for
`x'`y' portfolio."
			}

			local ++i

			restore, preserve
		}
		local i = `minYear'
		display in yellow "Done with `y'. Time $S_TIME at $S_DATE"
	}
	// Find arithmetic average for SH SM SL
	display in yellow "Done with `x'. Time $S_TIME at $S_DATE"
}

local endTime = c(current_time)
display in yellow "Program began at `beginTime' and ended at `endTime'
on $S_DATE."
clear all
discard

************* End of code****************


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/




--
Joachim Landström


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index