Package: vitals 0.2.0.9000

Simon Couch

vitals: Large Language Model Evaluation

A port of 'Inspect', a widely adopted 'Python' framework for large language model evaluation. Specifically aimed at 'ellmer' users who want to measure the effectiveness of their large language model-based products, the package supports prompt engineering, tool usage, multi-turn dialog, and model graded evaluations.

Authors:Simon Couch [aut, cre], Max Kuhn [ctb], Hadley Wickham [ctb], Mine Cetinkaya-Rundel [ctb], Posit Software, PBC [cph, fnd]

vitals_0.2.0.9000.tar.gz
vitals_0.2.0.9000.zip(r-4.7)vitals_0.2.0.9000.zip(r-4.6)vitals_0.2.0.9000.zip(r-4.5)
vitals_0.2.0.9000.tgz(r-4.6-any)vitals_0.2.0.9000.tgz(r-4.5-any)
vitals_0.2.0.9000.tar.gz(r-4.6-any)vitals_0.2.0.9000.tar.gz(r-4.5-any)
vitals_0.2.0.9000.tgz(r-4.5-emscripten)
vitals.pdf |vitals.html✨
vitals/json (API)
NEWS

# Install 'vitals' in R:

install.packages('vitals', repos = c('https://tidyverse.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/tidyverse/vitals/issues

Pkgdown/docs site:https://vitals.tidyverse.org

Datasets:

are - An R Eval

On CRAN:

7.63 score 52 stars 72 scripts 377 downloads 15 exports 36 dependencies

Last updated from:39b3f3db47. Checks:9 OK. Indexed: yes.

Target	Result	Time
linux-devel-x86_64	OK	160
source / vignettes	OK	231
linux-release-x86_64	OK	153
macos-release-arm64	OK	129
macos-oldrel-arm64	OK	121
windows-devel	OK	191
windows-release	OK	141
windows-oldrel	OK	101
wasm-release	OK	123

Exports:detect_answer detect_exact detect_includes detect_match detect_pattern generate generate_structured model_graded_fact model_graded_qa Task vitals_bind vitals_bundle vitals_log_dir vitals_log_dir_set vitals_view

Dependencies:askpass cli coro cpp11 curl dplyr ellmer fastmap generics glue httpuv httr2 jsonlite later lifecycle magrittr openssl otel pillar pkgconfig promises purrr R6 rappdirs Rcpp rlang S7 stringi stringr sys tibble tidyr tidyselect utf8 vctrs withr

Citation

Development and contributors

Readme and manuals

Help Manual

Help page	Topics
An R Eval	are
Convert a chat to a solver function	generate
Convert a chat to a solver function with structured output	generate_structured
Scoring with string detection	detect_answer detect_exact detect_includes detect_match detect_pattern scorer_detect
Model-based scoring	model_graded_fact model_graded_qa scorer_model
Creating and evaluating tasks	Task
Concatenate task samples for analysis	vitals_bind
Prepare logs for deployment	vitals_bundle
The log directory	vitals_log_dir vitals_log_dir_set
Interactively view local evaluation logs	vitals_view