Sufficient Computer Power to Handle Large Dataset (1 Viewer)

[lemo]

hi.
this forum is not exactly the place to post this question, as i'll be working mostly in SPSS and possibly R, but i thought i'll give it a shot anyway.

i have a computer with 2.4GHz Pentium 4 and 1G of RAM. i will soon need to work on a large dataset, let's say 100 million records, shipped in .CSV format. i heard the file size will be upwards of 100G.

do you think i have sufficient power to handle this?

thanks,
len

 

[boblarson]

I'm sure it can handle it, but at what cost (speed)? I would seriously bump your RAM up to 3Gb, if possible. 1Gb is hardly enough to deal with the OS and other overhead, let alone the data crunching.

Also, a hard drive upgrade if you have a 5400 RPM drive would be good to go to at least 7200 RPM, if not a 10000 RPM drive. The access times for the drive would be a good thing to check because with that kind of data, you will likely be doing a LOT of hard drive swap file reading/writing.

 

[lemo]

thanks bob.
don't think i can increase RAM to 3G, but will double-check.

so i think my strategy will be to go computer shopping, and pick a machine with your specs, but not buy it until i get the data and see how my current one handles it. if it slows to a halt, i'll be prepared to quickly go out and get a new one.

l

 

[boblarson]

Like I said, I think yours will handle it, but it may be slow in doing so. It all depends on how much patience you have

 

[lemo]

it's actually my wife who will do the bulk of work on this.
i think her patience is infinite..
l

 

[boblarson]

it's actually my wife who will do the bulk of work on this.
i think her patience is infinite..
l

Could be a good thing then

 

[lisamarietuck]

As a Newbie, I am always searching online for stuff that can help me. Thank you for your help.

 

[Atomic Shrimp]

As a Newbie, I am always searching online for stuff that can help me. Thank you for your help.

Really?! I always search for stuff I know will be useless.


You're a spammer, of course, but something went wrong, didn't it?

 

[boblarson]

Really?! I always search for stuff I know will be useless.


You're a spammer, of course, but something went wrong, didn't it?

Thanks AT that helped. They are now "outta here."