3 LLM Cost Model🔗ℹ

llm lang integrates several cost models of LLMs to report training and inference costs after each query. This feedback is reported after each prompt is sent to the llm-lang-logger as log-level/c 'info, topic 'llm-lang. The intention is to ensure the developer is given feedback about the potential resource cost of their query to encourage better prompt-engineering practice. The raw data is also available through current-carbon-use, current-power-use, and current-water-use.

The training costs are based on the work of [faiz2023]. A copy of the tooling, with additional data and estimates used to generate the training costs are included in the repository for llm lang.

The inference models are based on the work of [wilkins2024] and [wilkins2024a]. These estimates are even more rough, but unfortunately the best I can do with non-local LLMs.

model-name is used for debugging and reporting, and represents the model’s name to a developer.

query-tco2/kwh represent the metric tons of CO2 used to produce a KWh during an inference query. This information is specific the power generation for the power used to the language model during inference.

query-water-L/kWh represents the litres of water used per KWh during an inference query. This is the on-site scope 1 usage, which you can think of as the water consumptions of cooling aparatus in a data centre. It’s probably 0 for a local LLM. See [li2023] for more info.

training-water-L/kWh represents the litres of water used per KWh during training of the LLM. This is meant to be the on-site scope 1 usage.

training-tco2 represents the metric tons of CO2-equivalent emitted during training. This captures the power generation and emboddied carbon of the GPUs.

training-kwh represents the kilowatt-hours used to power computation while training the LLM.

inference-model represents the gen:kwh-model used for inferences to this LLM.

Bibliography🔗ℹ