Tech Requirements

Table of Contents

Phase 3: Blitz

💡
🤖🤖 note: this part is more technical; specifics ahead 🤖🤖

If we’re able to secure GPU’s & additional equipment, we can add exponential scale to both the creation & publication of the content. Here’s our teach leads dream scenario…

GPU Breakdown

We have two tasks: 1) fine-tuning and 2) generation/inference. We'll need both text generation via LLMs and image generation but the bottlenecks are mostly graphical so tailor performance based on image generation. We assume it takes 30 iterations to generate a single image.

Assuming that each additional fine-tuning design+prompt consists of 150 images+text, this is the estimated performance & cost breakdowns.

GPU
Estimated Time to Finetune
Inference Speed
Estimated Cost per Unit
nVidia H100 80gb pcie
15 minutes
27 it/s
$40,000
nVidia A100 80gb pcie
2 hours
47 it/s
$20,000
nVidia RTX 6000 Ada 48gb
>12 hours
42 it/s
$9,000
nVidia RTX 4090 24gb
>16 hours
33 it/s
$2,000

Caveat: RTX 4090's cannot be used in a data center configuration so cannot be scaled up to accommodate actual customers.

A-Tier Dream Scenario (Max Output / Cost)

  • 2x nVidia H100 80gb pcie
  • 8x nVidia RTX 6000 Ada
  • $152k value
  • 8-minute fine-tuning times
  • 8x concurrent designers

We use dual H100s for redundancy and since they do scale relatively linearly, it would also halve the fine-tuning time. If one dies, we can still do the processing. Any time not used to train/tune can be used for inference.

We use the cost effective RTX 6000 Adas for the generation but not to train.

B-Tier Dream Scenario

  • 2x nVidia A100 80gb pcie
  • 8x nVidia RTX 6000 Ada 48gb
  • $112k value
  • 1-hr fine tuning times
  • 8x concurrent designers

Same configuration as the S-Tier but with the lower cost A100s.

C-Tier Dream Scenario

  • 8x nVidia RTX 6000 Ada 48gb
  • $72k value
  • 1.5-hr fine tuning times if dedicated
  • 8x concurrent designers while not training

We use only the cost-effective inference setup and also train with it. It cannot train while generating and vice versa.

D-Tier Basic Scenario

  • 1x nvidia RTX 6000 Ada 48gb

========

  • $9k value
  • >12-hour fine tuning
  • Use personal 3090 homelab to do inference (2-3 concurrent designers)