ai2-olmo-eval0.7.1
ai2-olmo-eval0.7.1
Published
In-loop evaluation tasks for language modeling
pip install ai2-olmo-eval
Package Downloads
Requires Python
>=3.9
Dependencies
- numpy
<2.0 - torch
- torchmetrics
- datasets
- tokenizers
- scikit-learn
- cached-path
- requests
- packaging
- importlib_resources
- ruff
; extra == "dev" - mypy
<1.4,>=1.0; extra == "dev" - black
<24.0,>=23.1; extra == "dev" - isort
<5.13,>=5.12; extra == "dev" - pytest
; extra == "dev" - twine
>=1.11.0; extra == "dev" - setuptools
; extra == "dev" - wheel
; extra == "dev" - build
; extra == "dev" - boto3
; extra == "dev" - google-cloud-storage
; extra == "dev" - ai2-olmo-eval
[dev]; extra == "all"
OLMo-in-loop-evals
Code for in-loop evaluation tasks used by the OLMo training team.
Installation
pip install ai2-olmo-eval