Setting | Range to Try | Best Values | What It Does |
Context Window | 4 – 7 | 5 | How many previous words the AI considers. Below 3, it's messy. Above 7, the corpus is too small, so it backs off to smaller windows automatically. |
Temperature | 0.6 – 0.9 | 0.7 | How risky the AI's word choices are. Below 0.5, it repeats itself. Above 1.0, topics jump randomly. Sweet spot is 0.7. |
Text Length | 20 – 35 | 30 | How many words to generate. Longer text needs good context settings to stay coherent. |
Top-K | 3 – 10 (or 0 for “all”) | 5 | Only consider the top K candidates at each step. Low (1-3) = very focused, tiny pool. High = more variety, bigger pool. 0 = consider all words, which also works fine. |
Repetition Penalty | 0.3 – 1.0 | 0.5 | Discourages reusing words. 0 = words repeat freely. Above 1.0, output gets forced and unnatural. |
Starting Word | “The”, “In”, “Alice”, “By” | Any of these | Words that begin many sentences in the training data give the AI the most branching paths to follow. |
What You Tuned | What ML Engineers Tune | Scale Difference |
A few parameters | Billions of parameters. Learning rate, attention heads, layer count, embeddings, and thousands more. | You tuned a few dials. They tune billions. |
Context window of 1-10 words | 128,000 - 200,000+ tokens | Same concept. Wildly different scale. |
Temperature 0.1 - 2.0 | Temperature 0.0 - 2.0 | Identical range. Coding: 0.0-0.3. Creative writing: 0.8-1.2. |
Starting word to guide the topic | System prompt + conversation history | Your 1-word prompt vs. their thousands of words of instructions |
~4,000 words of training data | ~13 trillion tokens | Their data is about 3 billion times larger. |
Runs instantly in your browser | Thousands of GPUs, weeks of training | Training GPT-4 (which is old now) cost over $100 million. |