We introduced a new model designed for the Code generation task. Its test accuracy on the HumanEval base dataset surpasses that of GPT-4 Turbo (April 2024) and GPT-4o.

Self-evaluating interview for AI coders

Run evaluation on LLMs using human-eval benchmark

Evaluate LLM-generated COBOL

Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions

Code evaluation with *30-seconds-of-code* examples. Inspired by "Evaluating Large Language Models Trained on Code"