{"author":{"@type":"Person","name":"Alexander Bock","url":"https://abxk.ai/en/autor/alexander-bock/"},"categories":["AI Development"],"content":{"markdown":"In [your first API call](/en/ai-development/learn/your-first-llm-api-call/), the request named a placeholder model. In a real app, this one field is among your most important decisions. It sets the quality of the answers, how long users wait and what each request costs. There are many candidates: our [LLM database](/en/llm/) alone lists {{\u003c stat \"models\" \u003e}} models.\n\nThis is part 4 of Level 1 in our AI Development learning path. You will learn which criteria matter and how they pull against each other. You will also see what open and closed weights mean for your app and how to test a shortlist on your own data. It helps to know the basics of tokens and context windows from [Level 0](/en/ai-development/learn/apis-tokens-and-context-windows/).\n\n## Start with your task, not with a ranking\n\nRankings show which model is strongest on average across many tests. Your app needs something else: a model that does your task well enough, fast enough and at a cost you can carry, under your data rules. That is often not the model at the top of a ranking.\n\nSo write down two things before you look at any model:\n\n- **The task in one sentence**, for example \"Answer customer questions from our help articles in English and German.\"\n- **The hard requirements**: things a model must meet, or it is out. Must the data stay in the EU or on your own servers? Which languages? How long are the inputs? Will users upload images or PDF files? How quickly must the answer start?\n\nWith these notes, you can ask the same six questions of every candidate:\n\n![Six cards with the criteria for choosing a model: quality measured on your own data, speed of the first word and the full answer, cost of input and output tokens per request, context window, open or closed weights, and where data is processed and stored](/images/learn/en/choose-a-model-for-your-app-criteria.webp \"Six questions for every candidate model. Quality comes first, measured on your own data.\")\n\n## Quality, speed and cost pull against each other\n\n**Quality** means how well a model solves your task. General benchmarks give a first idea. A useful overview is the **Epoch Capabilities Index** (ECI), which combines the results of many benchmarks into one number per model. These are the current leaders in our data:\n\n{{\u003c llm-top bench=\"eci\" n=\"8\" \u003e}}\n\nA high rank tells you that a model is strong in general. It does not tell you how well it handles your support tickets, your product data or your language. Only a test with your own inputs can show that — the last section explains how.\n\n**Speed**, also called **latency**, has two parts. The **time to first token** is how long it takes until the answer starts. The **output speed** is how many tokens per second the model writes after that. Large models and **reasoning models**, which work through a problem step by step before they answer, are often slower. With **streaming**, users see the answer while it is written, so the wait feels shorter.\n\n**Cost** depends on the tokens of each request. APIs bill input and output tokens separately, with different prices for each model. A larger model usually costs more per token, and reasoning adds output tokens that you also pay for. Compare the cost of a typical request with your own token counts, not only the price list.\n\nWithin a model family, these three criteria usually move together: the larger model is stronger, slower and more expensive.\n\n![Comparison of a large frontier model and a small, fast model: the large one is stronger on hard tasks but slower and more expensive per token; the small one is often good enough for clear, narrow tasks, answers quickly and costs less; below, the rule of thumb to test with a strong model first and then pick the smallest model that passes](/images/learn/en/choose-a-model-for-your-app-size.webp \"Large or small model? Typical tendencies within a model family. Your tests decide.\")\n\nThe **context window** is the fourth criterion: it has to hold your instructions, documents, chat history and the answer. A larger window is not automatically better, because every token you send costs money and time.\n\n## Open or closed weights\n\nThe **weights** are the numbers a model learned during training. With **open weights**, the model files are published. You can run the model on your own hardware or on rented GPUs, and many hosting services also offer such models through an API. With **closed weights**, you can only use the model through the provider's API or app. In our database, {{\u003c stat \"models-open\" \u003e}} models have open weights and {{\u003c stat \"models-closed\" \u003e}} are closed.\n\n![Two charts from our LLM database: the share of models with open weights per release year as bars, and the top 30 models of the overall index ECI as a strip of squares colored by open or closed weights, with the best open-weight model marked](/images/learn/en/choose-a-model-for-your-app-open-closed.webp \"Open and closed models in our LLM database. The values come straight from the data. Data: Epoch AI, CC BY 4.0.\")\n\nThe figure shows two views of the same data: what share of the new models in our database came with open weights each year, and where open-weight models stand in the overall ranking. Our database lists models with published test results, so it shows these models, not every model on the market. Here are a few strong open-weight models from our data:\n\n{{\u003c llm-models \"kimi-k3 deepseek-v4-1-flash deepseek-v4-pro-0813 glm-5-3-flash qwen-3-8-27b\" \u003e}}\n\nA number such as 27B in a model name usually gives the number of **parameters**, here 27 billion. As a rough guide, more parameters need more GPU memory and more computing power.\n\nThe choice between the two groups is about control and effort:\n\n| | Open weights | Closed weights |\n|---|---|---|\n| Where it runs | Where you decide: your servers, a GPU cloud, a hosting service | At the provider |\n| Your data | Can stay in your own environment | Goes to the provider, under its terms |\n| Versions | You keep the files as long as you like | The provider updates and retires versions |\n| Adapting | Fine-tuning is possible on your own hardware | Only where the provider offers it |\n| Effort | You run, scale and secure the setup — or pay a host | Little: an API key is enough |\n| License | Varies; some licenses restrict certain uses | Set by the provider's terms of service |\n\n\"Open weights\" does not always mean \"open source\". Some licenses limit commercial use, set conditions for very large services or require you to name the model. Read the license before you build on a model. If you want to run an open-weight model yourself, a GPU cloud such as [Runpod](/en/ai-development/#tool-runpod) rents the hardware, billed by usage according to the vendor. Level 3 explains [when renting GPUs makes sense](/en/ai-development/learn/gpu-infrastructure-for-ai/).\n\n## Questions to ask before you commit\n\nBenchmarks do not answer the questions about data and operations. Find the answers in the provider's documentation, terms and data processing agreement:\n\n| Question | Why it matters |\n|---|---|\n| Are your API inputs used to train models? | Inputs may contain customer data or trade secrets |\n| Where is data processed and stored? | Some data rules require a region, such as the EU |\n| How long are inputs and outputs kept? | Shorter storage means less risk |\n| Is there a data processing agreement? | Under the GDPR, a provider that processes personal data on your behalf needs one (Article 28) |\n| Can you use a fixed model version, and when will it be retired? | A silent change or a retirement can break your app |\n| Does the model support what your app needs? | For example JSON output, tool calling, images or your languages |\n\n## Test your shortlist on your own data\n\nNow narrow the field. Drop every model that fails a hard requirement, and pick two to four candidates of different sizes — ideally from at least two providers. Then test them all with the same prompt.\n\n![A funnel with five stages: all candidates in our LLM database and the providers' lists, hard requirements, a shortlist of two to four models, a test on 20 to 50 real inputs, and the decision for the smallest model that passes, plus a fallback](/images/learn/en/choose-a-model-for-your-app-shortlist.webp \"From many models to one: hard requirements first, then a test with your own inputs.\")\n\n1. **Collect 20 to 50 real inputs** with the result you expect for each. Include easy, typical and tricky cases. If you versioned your prompts as shown in part 2, you already have such a test set.\n2. **Run every candidate** with the same prompt and settings, and record the answer, the time and the tokens of each request.\n3. **Score the answers.** Mark each one as correct or not, or use a simple scale. Level 2 turns this into automated [evals](/en/ai-development/learn/test-ai-features-with-evals/).\n4. **Calculate the cost** per 1,000 requests from the token counts and the provider's current prices.\n5. **Decide.** Choose the smallest or cheapest model that meets your quality bar. Keep the runner-up as a **fallback**, ideally from another provider, for outages.\n\nA short script does steps 2 and 4 for you. It uses the request from part 1 and writes one row per answer into a CSV file that you can open in a spreadsheet:\n\n```python\nimport csv\nimport os\nimport time\n\nimport requests\n\nURL = \"https://api.example.com/v1/chat\"                  # your provider's endpoint\nHEADERS = {\"Authorization\": f\"Bearer {os.environ['LLM_API_KEY']}\"}\nCANDIDATES = [\"model-large\", \"model-medium\", \"model-small\"]  # model IDs from the docs\nSYSTEM = open(\"prompts/faq_v3.txt\").read()               # the same prompt for every model\n\nwith open(\"tests.csv\", newline=\"\") as f:                 # columns: question, expected\n    tests = list(csv.DictReader(f))\n\nwith open(\"results.csv\", \"w\", newline=\"\") as f:\n    out = csv.writer(f)\n    out.writerow([\"model\", \"question\", \"expected\", \"answer\", \"seconds\", \"tokens_in\", \"tokens_out\"])\n    for model in CANDIDATES:\n        for t in tests:\n            start = time.perf_counter()\n            r = requests.post(URL, headers=HEADERS, timeout=60, json={\n                \"model\": model,\n                \"messages\": [{\"role\": \"system\", \"content\": SYSTEM},\n                             {\"role\": \"user\", \"content\": t[\"question\"]}],\n                \"max_tokens\": 400,\n            })\n            r.raise_for_status()\n            data = r.json()                              # layout as in part 1; it varies by provider\n            out.writerow([model, t[\"question\"], t[\"expected\"], data[\"output\"][0][\"text\"],\n                          round(time.perf_counter() - start, 2),\n                          data[\"usage\"][\"input_tokens\"], data[\"usage\"][\"output_tokens\"]])\n```\n\nRun each test more than once if answers vary, and look at the slowest answers, not only at the average. A model that is fast most of the time but sometimes takes much longer can still frustrate your users.\n\n{{\u003c callout type=\"tip\" title=\"Keep the model name out of your code\" \u003e}}\nStore the model ID in a setting, for example an environment variable such as `LLM_MODEL`. Then you can switch to your fallback, test a new model or roll back without changing code. Log the model version with every response, so you know which model wrote which answer.\n{{\u003c /callout \u003e}}\n\n## Which model for which app: a decision table\n\nYour test results decide, but this table gives you a sensible starting point for the shortlist:\n\n| Your app | Start with | Watch out for |\n|---|---|---|\n| Answers from your documents (FAQ, support) | A small or mid-size model with enough context for your passages | Many wrong answers come from the search, not the model — check the passages first |\n| Sorting, tagging, extracting data as JSON | A small, fast model | Edge cases; check the JSON in code |\n| Drafts and summaries that people edit | A mid-size model | Tone and length; a person still reviews |\n| Coding help, complex analysis, multi-step agents | A model near the top of the overall ranking | Time and cost per task |\n| Live chat or voice with short replies | A fast model, with streaming | Time to first token |\n| Data that must stay on your servers | An open-weight model on your own or rented hardware | License terms; the effort of running GPUs |\n| Long contracts or manuals | A model with a large context window — or a search over your documents instead | Cost per request; details lost in long inputs |\n\nCheck your choice again from time to time. New models appear often, and providers retire older versions. Because your test set and script already exist, checking a new candidate takes an hour, not a project.","text":"In your first API call, the request named a placeholder model. In a real app, this one field is among your most important decisions. It sets the quality of the answers, how long users wait and what each request costs. There are many candidates: our LLM database alone lists 343 models.\nThis is part 4 of Level 1 in our AI Development learning path. You will learn which criteria matter and how they pull against each other. You will also see what open and closed weights mean for your app and how to test a shortlist on your own data. It helps to know the basics of tokens and context windows from Level 0.\nStart with your task, not with a ranking Rankings show which model is strongest on average across many tests. Your app needs something else: a model that does your task well enough, fast enough and at a cost you can carry, under your data rules. That is often not the model at the top of a ranking.\nSo write down two things before you look at any model:\nThe task in one sentence, for example \u0026ldquo;Answer customer questions from our help articles in English and German.\u0026rdquo; The hard requirements: things a model must meet, or it is out. Must the data stay in the EU or on your own servers? Which languages? How long are the inputs? Will users upload images or PDF files? How quickly must the answer start? With these notes, you can ask the same six questions of every candidate:\nSix questions for every candidate model. Quality comes first, measured on your own data. Quality, speed and cost pull against each other Quality means how well a model solves your task. General benchmarks give a first idea. A useful overview is the Epoch Capabilities Index (ECI), which combines the results of many benchmarks into one number per model. These are the current leaders in our data:\nEpoch Capabilities Index: the top 8 models #ModelReleasedResult 1Claude Opus 5.5 AnthropicSep 22, 2026166.8 2GPT-6 Astra OpenAISep 3, 2026166.4 3GPT-6.1 Sol OpenAISep 29, 2026166.3 4Claude Fable 5.1 AnthropicSep 1, 2026164.7 5Gemini 4 Argon Google DeepMindSep 30, 2026164.6 6Claude Sonnet 5.5 AnthropicSep 28, 2026164.4 7Claude Opus 5 AnthropicJul 24, 2026162.6 8GPT-6 Sol OpenAISep 22, 2026162.6 Combines the results of more than 50 benchmarks into one number per model, so that models can be compared even if they were never tested on the same benchmarks. It is an index, not a percentage: Epoch AI scales it so that Claude 3.5 Sonnet stands at 130 and GPT-5 at 150. Data: Epoch AI, retrieved on October 17, 2026, licensed CC BY 4.0. All models: LLM database.\nA high rank tells you that a model is strong in general. It does not tell you how well it handles your support tickets, your product data or your language. Only a test with your own inputs can show that — the last section explains how.\nSpeed, also called latency, has two parts. The time to first token is how long it takes until the answer starts. The output speed is how many tokens per second the model writes after that. Large models and reasoning models, which work through a problem step by step before they answer, are often slower. With streaming, users see the answer while it is written, so the wait feels shorter.\nCost depends on the tokens of each request. APIs bill input and output tokens separately, with different prices for each model. A larger model usually costs more per token, and reasoning adds output tokens that you also pay for. Compare the cost of a typical request with your own token counts, not only the price list.\nWithin a model family, these three criteria usually move together: the larger model is stronger, slower and more expensive.\nLarge or small model? Typical tendencies within a model family. Your tests decide. The context window is the fourth criterion: it has to hold your instructions, documents, chat history and the answer. A larger window is not automatically better, because every token you send costs money and time.\nOpen or closed weights The weights are the numbers a model learned during training. With open weights, the model files are published. You can run the model on your own hardware or on rented GPUs, and many hosting services also offer such models through an API. With closed weights, you can only use the model through the provider\u0026rsquo;s API or app. In our database, 172 models have open weights and 160 are closed.\nOpen and closed models in our LLM database. The values come straight from the data. Data: Epoch AI, CC BY 4.0. The figure shows two views of the same data: what share of the new models in our database came with open weights each year, and where open-weight models stand in the overall ranking. Our database lists models with published test results, so it shows these models, not every model on the market. Here are a few strong open-weight models from our data:\nModels from our LLM database ModelReleasedWeightsECIRank Kimi K3 MoonshotJul 16, 2026open157.417 DeepSeek V4.1 Flash DeepSeekSep 9, 2026open155.631 DeepSeek V4 Pro 0813 DeepSeekAug 13, 2026open155.332 GLM-5.3-Flash Z.ai (Zhipu AI)Aug 20, 2026open153.245 Qwen 3.8 27B AlibabaAug 14, 2026open151.652 ECI = Epoch Capabilities Index, one overall score across many benchmarks; rank among all models with an ECI. Data: Epoch AI, retrieved on October 17, 2026, licensed CC BY 4.0. All models: LLM database.\nA number such as 27B in a model name usually gives the number of parameters, here 27 billion. As a rough guide, more parameters need more GPU memory and more computing power.\nThe choice between the two groups is about control and effort:\nChoose a Model for Your App: Quality, Speed, Cost, Openness Open weights Closed weights Where it runs Where you decide: your servers, a GPU cloud, a hosting service At the provider Your data Can stay in your own environment Goes to the provider, under its terms Versions You keep the files as long as you like The provider updates and retires versions Adapting Fine-tuning is possible on your own hardware Only where the provider offers it Effort You run, scale and secure the setup — or pay a host Little: an API key is enough License Varies; some licenses restrict certain uses Set by the provider\u0026rsquo;s terms of service \u0026ldquo;Open weights\u0026rdquo; does not always mean \u0026ldquo;open source\u0026rdquo;. Some licenses limit commercial use, set conditions for very large services or require you to name the model. Read the license before you build on a model. If you want to run an open-weight model yourself, a GPU cloud such as Runpod rents the hardware, billed by usage according to the vendor. Level 3 explains when renting GPUs makes sense.\nQuestions to ask before you commit Benchmarks do not answer the questions about data and operations. Find the answers in the provider\u0026rsquo;s documentation, terms and data processing agreement:\nChoose a Model for Your App: Quality, Speed, Cost, Openness (2) Question Why it matters Are your API inputs used to train models? Inputs may contain customer data or trade secrets Where is data processed and stored? Some data rules require a region, such as the EU How long are inputs and outputs kept? Shorter storage means less risk Is there a data processing agreement? Under the GDPR, a provider that processes personal data on your behalf needs one (Article 28) Can you use a fixed model version, and when will it be retired? A silent change or a retirement can break your app Does the model support what your app needs? For example JSON output, tool calling, images or your languages Test your shortlist on your own data Now narrow the field. Drop every model that fails a hard requirement, and pick two to four candidates of different sizes — ideally from at least two providers. Then test them all with the same prompt.\nFrom many models to one: hard requirements first, then a test with your own inputs. Collect 20 to 50 real inputs with the result you expect for each. Include easy, typical and tricky cases. If you versioned your prompts as shown in part 2, you already have such a test set. Run every candidate with the same prompt and settings, and record the answer, the time and the tokens of each request. Score the answers. Mark each one as correct or not, or use a simple scale. Level 2 turns this into automated evals. Calculate the cost per 1,000 requests from the token counts and the provider\u0026rsquo;s current prices. Decide. Choose the smallest or cheapest model that meets your quality bar. Keep the runner-up as a fallback, ideally from another provider, for outages. A short script does steps 2 and 4 for you. It uses the request from part 1 and writes one row per answer into a CSV file that you can open in a spreadsheet:\nimport csv import os import time import requests URL = \u0026#34;https://api.example.com/v1/chat\u0026#34; # your provider\u0026#39;s endpoint HEADERS = {\u0026#34;Authorization\u0026#34;: f\u0026#34;Bearer {os.environ[\u0026#39;LLM_API_KEY\u0026#39;]}\u0026#34;} CANDIDATES = [\u0026#34;model-large\u0026#34;, \u0026#34;model-medium\u0026#34;, \u0026#34;model-small\u0026#34;] # model IDs from the docs SYSTEM = open(\u0026#34;prompts/faq_v3.txt\u0026#34;).read() # the same prompt for every model with open(\u0026#34;tests.csv\u0026#34;, newline=\u0026#34;\u0026#34;) as f: # columns: question, expected tests = list(csv.DictReader(f)) with open(\u0026#34;results.csv\u0026#34;, \u0026#34;w\u0026#34;, newline=\u0026#34;\u0026#34;) as f: out = csv.writer(f) out.writerow([\u0026#34;model\u0026#34;, \u0026#34;question\u0026#34;, \u0026#34;expected\u0026#34;, \u0026#34;answer\u0026#34;, \u0026#34;seconds\u0026#34;, \u0026#34;tokens_in\u0026#34;, \u0026#34;tokens_out\u0026#34;]) for model in CANDIDATES: for t in tests: start = time.perf_counter() r = requests.post(URL, headers=HEADERS, timeout=60, json={ \u0026#34;model\u0026#34;: model, \u0026#34;messages\u0026#34;: [{\u0026#34;role\u0026#34;: \u0026#34;system\u0026#34;, \u0026#34;content\u0026#34;: SYSTEM}, {\u0026#34;role\u0026#34;: \u0026#34;user\u0026#34;, \u0026#34;content\u0026#34;: t[\u0026#34;question\u0026#34;]}], \u0026#34;max_tokens\u0026#34;: 400, }) r.raise_for_status() data = r.json() # layout as in part 1; it varies by provider out.writerow([model, t[\u0026#34;question\u0026#34;], t[\u0026#34;expected\u0026#34;], data[\u0026#34;output\u0026#34;][0][\u0026#34;text\u0026#34;], round(time.perf_counter() - start, 2), data[\u0026#34;usage\u0026#34;][\u0026#34;input_tokens\u0026#34;], data[\u0026#34;usage\u0026#34;][\u0026#34;output_tokens\u0026#34;]]) Run each test more than once if answers vary, and look at the slowest answers, not only at the average. A model that is fast most of the time but sometimes takes much longer can still frustrate your users.\n✦Keep the model name out of your code\nStore the model ID in a setting, for example an environment variable such as LLM_MODEL. Then you can switch to your fallback, test a new model or roll back without changing code. Log the model version with every response, so you know which model wrote which answer.\nWhich model for which app: a decision table Your test results decide, but this table gives you a sensible starting point for the shortlist:\nChoose a Model for Your App: Quality, Speed, Cost, Openness (3) Your app Start with Watch out for Answers from your documents (FAQ, support) A small or mid-size model with enough context for your passages Many wrong answers come from the search, not the model — check the passages first Sorting, tagging, extracting data as JSON A small, fast model Edge cases; check the JSON in code Drafts and summaries that people edit A mid-size model Tone and length; a person still reviews Coding help, complex analysis, multi-step agents A model near the top of the overall ranking Time and cost per task Live chat or voice with short replies A fast model, with streaming Time to first token Data that must stay on your servers An open-weight model on your own or rented hardware License terms; the effort of running GPUs Long contracts or manuals A model with a large context window — or a search over your documents instead Cost per request; details lost in long inputs Check your choice again from time to time. New models appear often, and providers retire older versions. Because your test set and script already exist, checking a new candidate takes an hour, not a project.\n","words":1935},"date":"2026-10-14","description":"How to pick a language model for your app: weigh quality, speed, cost, context and openness, then test a shortlist on your own data.","image":"https://abxk.ai/images/cover/choose-a-model-for-your-app.webp","json":"https://abxk.ai/en/ai-development/learn/choose-a-model-for-your-app/index.json","kind":"page","language":"en","learning_path":{"category":"ai-development","level":1,"level_name":"Beginner"},"modified":"2026-10-14","related_tools":["Runpod","MindStudio","Devs.ai"],"section":"ai-development","seo_title":"Choose a Model for Your App: Quality, Speed, Cost","tags":["model-selection","llm","open-weights","evaluation"],"title":"Choose a Model for Your App: Quality, Speed, Cost, Openness","translations":[{"json":"https://abxk.ai/de/ai-development/learn/choose-a-model-for-your-app/index.json","language":"de","url":"https://abxk.ai/de/ai-development/learn/choose-a-model-for-your-app/"}],"url":"https://abxk.ai/en/ai-development/learn/choose-a-model-for-your-app/"}