Well, partly. I am using a technique called fine-tuning, where I take GPT2, pick up where it left off, and train it more with my own data. This is because GPT2 already understands the semantics and grammar of English so now I just keep training it to adapt to a new vocab set (which is this case is a lot of misspellings and slang).
1 Like