Integration with the Hugging Face Hub

Why share to the Hugging Face Hub

The Hub is a central platform where anyone can share and explore models, datasets, and ML demos. It aims to build the most extensive collection of Open Source models, datasets, and demos.

Sharing to the Hub could amplify the impact of a fastai Learner by making it available for others to download and explore.

Anyone can access all the fastai models in the Hub by filtering the huggingface.co/models webpage by the fastai library, as in the image below.

To know more about the Hub, refer to this introduction.

hf_hub_fastai

Installation

Install huggingface_hub. Additionally, the integration functions require the following packages:

  • toml,
  • fastai>=2.4,
  • fastcore>=1.3.27

You can install these packages manually or specify ["fastai"] when installing huggingface_hub, and your environment will be ready:

pip install huggingface_hub["fastai"]

Note: As of May 5, 2022, there has not been a release that includes the fastai+hub features so that you can install from main:

!pip install git+https://github.com/huggingface/huggingface_hub#egg=huggingface-hub["fastai"]

To share models in the Hub, you will need to have a user. Create it on the Hugging Face website.

Sharing a Learner to the Hub

There are three options to login to the Hugging Face Hub; your token will be available in your Account Settings:

  1. Type huggingface-cli login in your terminal and enter your token.

  2. If in a python notebook, you can use notebook_login.

from huggingface_hub import notebook_login
notebook_login()
  1. Use the token argument of the push_to_hub_fastai function.

Input push_to_hub_fastai (refer to the Hub Client documentation) with the Learner you want to upload and the repository id for the Hub in the format of "namespace/repo_name". The namespace can be an individual account or an organization you have write access to (for example, 'fastai/stanza-de'). For more details, refer to the Hub Client documentation.

from huggingface_hub import push_to_hub_fastai
push_to_hub_fastai(learner=learn, repo_id="espejelomar/cool_learner")

The Learner is now in the Hub with the id espejelomar/cool_learner.

When uploading a fastai Learner (or any other model) to the Hub, it is helpful to edit its model card (image below) so that others better understand your work (refer to the Hugging Face documentation).

hf_model_card

Loading a Learner from Hub

To load a model in the Hub with the id ITESM/fastai_model:

from huggingface_hub import from_pretrained_fastai
learner = from_pretrained_fastai("ITESM/fastai_model")

The Hub Client documentation includes addtional details on from_pretrained_fastai.