Need a new read.table

[bwlewis]

I'm having trouble loading the airline data with read.table, even using xmem. R is making tons of tiny allocations.

I'm thinking of something with read.table syntax (like lazy.frame) that:

1 quickly inspects the file for number of lines and variable types 2 allocates a list of big variables 3 fills in the allocated variables by iterating through the file row-wise

this approach should use xmem reasonably well I think.