Open Source Tools

Our tools

Powerful, production-ready tools to streamline your AI data workflows. Built by researchers, for researchers.

Tahdheeb

LLM Data Preprocessing & Cleaning

A powerful tool to preprocess and clean LLM training data, ensuring your datasets are optimized and ready for training large language models.

Key Features:

Data validation and quality checks
Format standardization
Duplicate detection and removal
Text normalization and cleaning
Pipeline automation
Export ready-to-train datasets

View on GitHub Documentation

Data Factory

Synthetic Data Generation

A comprehensive library for generating high-quality synthetic data, enabling rapid prototyping and augmenting training datasets with diverse, realistic examples.

Key Features:

Multiple generation strategies
Customizable data templates
Quality control mechanisms
Scalable generation pipelines
Integration with popular frameworks
Evaluation metrics

View on GitHub Documentation

Want to Contribute?

These tools are open source and community-driven. We welcome contributions, feedback, and collaboration from developers worldwide.

Explore Repositories Join Community