19 May 2013

EC2, Amleto and LightWave

Amazon's EC2 service can be used as a LightWave renderfarm, simply and cost effectively. I will write up how it can be done, right now, just notes.

  • Elastic IP not needed.
  • Make sure instances have enough memory for the scene.
  • Amleto needs the master's IP address UDP does not work
  • Add extra drive to basic instance to act as content drive
  • install Dropbox and VNC
  • When creating spot requests name them with the last bits of the IP address to better identify them (eg. 56.74.243.12 gets named 243.12)
  • The security group needs stuff opened up, but I want to close ports that aren't needed down to minimize security risks
  • Terminate instances using spot requests and cancel the requests otherwise you pay
  • keep making AMIs as you refine your farm
  • Add Amleto Server and Client to Startup folder
  • Autologon for render nodes:Autologon

okay, so for a renderfarm you just need to install a LightWave trial edition on your first instance, which can be a t1.micro machine, it doesn't need to be bigger (except you might needs an EBS hard drive). Install Amleto (the render controller), Dropbox and umm, that's about it. Make an AMI from that setup and call it Farmer.

Then launch it again but this time load the Amleto client (and make sure it's in Startup) and Dropbox and Mark Russinovich's Autologon so that once you start those instances they begin rendering immediately.

Create an image of that setup and call it node and then you can request as many spot instances of that as you need and just view the Farmer computer and watch them come online and start rendering.

(Slightly updated 08.Jan.14 to make it a bit clearer)

The table below is based on 32/64-bit Windows machines and sorted on ECU, which is a measure of how rapid a node is. As you can see the t1.micro you will use as Farmer is not expensive to run a roughly 3 cents/hour. What you use for your nodes is up to you. The c machines are the ones designed for computing and I've highlighted a few good examples. Amazon updates the offerings quite regularly so this table will go out of date quite quickly I imagine.
 
vCPUECUMemory (GiB)Instance Storage (GB)Windows Usage
t1.micro1Variable0.615EBS Only$0.035 per Hour
m1.small111.71 x 160$0.091 per Hour
m1.medium123.751 x 410$0.182 per Hour
c1.medium251.71 x 350$0.225 per Hour
c3.large273.752 x 16 SSD$0.233 per Hour
m1.large247.52 x 420$0.364 per Hour
c3.xlarge4147.52 x 40 SSD$0.466 per Hour
m2.xlarge26.517.11 x 420$0.510 per Hour
m3.xlarge413152 x 40 SSD$0.702 per Hour
m1.xlarge48154 x 420$0.728 per Hour
g2.2xlarge8261560 SSD$0.767 per Hour
c1.xlarge82074 x 420$0.900 per Hour
c3.2xlarge828152 x 80 SSD$0.932 per Hour
i2.xlarge41430.51 x 800 SSD$0.973 per Hour
m2.2xlarge41334.21 x 850$1.020 per Hour
m3.2xlarge826302 x 80 SSD$1.404 per Hour
c3.4xlarge1655302 x 160 SSD$1.864 per Hour
i2.2xlarge827612 x 800 SSD$1.946 per Hour
m2.4xlarge82668.42 x 840$2.040 per Hour
cg1.4xlarge1633.522.52 x 840$2.600 per Hour
cc2.8xlarge328860.54 x 840$2.970 per Hour
hi1.4xlarge163560.52 x 1024 SSD$3.580 per Hour
c3.8xlarge32108602 x 320 SSD$3.728 per Hour
cr1.8xlarge32882442 x 120 SSD$3.831 per Hour
i2.4xlarge16531224 x 800 SSD$3.891 per Hour
hs1.8xlarge163511724 x 2048$4.931 per Hour
i2.8xlarge321042448 x 800 SSD$7.782 per Hour