Documentation
Quick-start guides
Load datasets, transcribe speech, and run the Somali tokenizer.
All datasets and models are hosted on Hugging Face under the goobolabs organisation. Install with: pip install datasets transformers.
Load a dataset
from datasets import load_dataset
# Load the Somali text corpus
ds = load_dataset("goobolabs/somnlp-corpus")
print(ds["train"][0])
# {'text': 'Waxaan jeclahay barashada...', 'domain': 'news', 'dialect': 'standard'}Transcribe Somali speech (ASR)
import whisper
model = whisper.load_model("goobolabs/whisper-som-small")
result = model.transcribe("audio.wav", language="so")
print(result["text"])
# 'Beeraha Soomaaliya waxay u baahan yihiin...'Tokenize Somali text
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("goobolabs/goobo-tokenizer")
tokens = tok.tokenize("Caafimaadka carruurtu waa mudnaanteenna koowaad.")
print(tokens)
# ['Caafimaad', '##ka', 'carruur', '##tu', 'waa', ...]Frequently asked
Are the datasets and models free to use?
Yes. Everything here is hosted openly on Hugging Face under the goobolabs organisation. License terms vary by release, so check the license listed on each dataset or model card before using it in production.
Which models are available right now?
whisper-som-small for speech recognition, somast-tokenizer for Somali-native tokenization, and som-en-mt for Somali-English translation. See the Models section for details on each.
Newsletter
Stay in the loop
Dataset releases, new bootcamp cohorts, research papers, and lab updates, delivered straight to your inbox. No spam, unsubscribe anytime.