![]() ![]() This notebook does the same except that instead of loading data from HuggingFace dataset API, it reads the data from CSV files.The final model is saved to a directory named 'outputEOS'.The objective is to create independency between each text.This notebook does the same as notebooks/FineTuneGPT2.ipynb except it adds an token at the end of each text before concatenating.The fine-tuned model is then used to generate bank domain specific texts.The final model is saved to a directory named 'output'.The model is only trained over 3 epochs only (For debugging purposes or quicker training).The dataset is tokenized using the gpt2 tokenizer, all texts from the dataset are then concatenated to generate chunks of block_size of 1024.The dataset comprises 13,083 customer service queries (For debugging purposes or quicker training we use a small dataset).The objective is to create a model that will generate a domain specific language (banking domain in this case).The dataset is loaded using HuggingFace's dataset API.This notebook will finetune the pretrained GPT2 model using tensorflow on the banking77 dataset:.Checks out how we can use them using huggingface's pipeline class & Tensorflow model specific classes to predict next token or generate texts.Checks out the different API versions offered by HuggingFace.Runs them over a list of examples to show the results.Ĭasual Language Modeling: Shows how we can generate sequences' next token or texts.Checks out how we can use them using huggingface's pipeline class & Tensorflow model specific classes to predict masked tokens.Explains BERT, RoBERTa & CamemBERT models.Masked Language Modeling: Shows how we can perform Masked Language Modeling and Next Sentence Prediction (using pretrained models in HuggingFace) Notebooks/LanguageModelingPretrained.ipynb: Notebooks: contains all notebooks that explore the use of huggingface pretrained APIs, and how to finetune the pretrained models locally over custom data with Tensorflow and how to use finetuned models to generate next token or next texts. This repo explores the use of pretrained NLP models for Language Modeling offered by HuggingFace, shows how to finetune the pretrained model over your data, and how to train and deploy using Azure Machine Learning Pipelines. ![]()
0 Comments
Leave a Reply. |