MemoryError during Python list processing -

(python version 2.6.5)

i have:

 boxes_with_sizes_added = [\  [0,0,0,1,1,1,0],\  [785,500,200,787,502,202,1],\  [400,500,600,404,504,604,2],\  [100,200,300,108,208,308,3],\  [50,60,70,51,61,71,0]\  # several millions more...  ] 

...they boxes in format: [x1,y1,z1,x2,y2,z2,rel_size]

and have 'chopping' method:

def cubic_breakdown(box,division_factor):  if division_factor==1:      return[box]  elif division_factor>1:      boxes_out=[]      k in range(division_factor):          j in range(division_factor):              in range(division_factor):                  boxes_out.append([\                  (box[0]+((box[3]-box[0])/float(division_factor))*i),\                  (box[1]+((box[4]-box[1])/float(division_factor))*j),\                  (box[2]+((box[5]-box[2])/float(division_factor))*k),\                  (box[0]+((box[3]-box[0])/float(division_factor))*(i+1)),\                  (box[1]+((box[4]-box[1])/float(division_factor))*(j+1)),\                  (box[2]+((box[5]-box[2])/float(division_factor))*(k+1)),\                  box[6]\                  ])      return boxes_out 

where 'box' 'chopped' equal segments according it's 'rel_size' , added list

 chopped_boxes=[]  box in boxes_with_sizes_added:      chopped_box in cubic_breakdown(box,2**box[6]):          chopped_boxes.append(chopped_box) 

when try process many boxes, however, 'memoryerror' @ point. problem? need pickle list or list output? in advance!

for each item create list (boxes_out) 6 * division_factor ** 3 elements, , every element of input. division_factor = 2 you're increasing size of data 48 times. don't know how ram have chances it's not enough.

• try using numpy arrays; these more compact , efficient, might enough fit data in ram.
• try using database, e.g sqlite, , store data on disk. algorithm looks sequential, don't seem need data in ram @ same time.
• get bigger machine :) no, really, renting high-memory ec2 instance $0.5 $1.5 hour, , these come plenty of ram.