Language models aim to create intuitive human-computer interfaces, but this is difficult because language syntax doesn't fully capture semantic meaning.
Deep learning works by performing a series of transformations on input data to warp its vector space until it becomes linearly separable.
The training objective, or incentive, dictates exactly what a model learns and can lead to unintended outcomes if not designed carefully.
The evolution from slow, non-contextual models like LSTMs to the parallel and deeply contextual transformer architecture solved major NLP challenges.
A scaled-down, character-level transformer model demonstrates the 'fill in the blank' pre-training task by predicting masked characters in artist names.
By analyzing a model's internal weights, we can see it learns phonetic relationships and syntactic structures without ever being explicitly trained on them.
Generative models like GPT are excellent at predicting the next word based on statistical patterns but lack the underlying thought process required for true, creative writing.
Blank Language Models (BLM) offer a new training approach by filling in text in any order, forcing the model to consider both past and future context.
The complexity of implementing novel architectures like BLMs highlights the need for better infrastructure and compiled languages like Swift for TensorFlow to speed up innovation.
58:00Vijay Krishan Gupta & Gauravdeep Singh Lotey
29:40Jodie Burchell
35:16Ankit Patel
23:50Aditi Godbole
56:55Ekaterina Sirazitdinova
25:17Scott Hanselman
32:54Adrian Spataru & Bohdan Andrusyak
52:37Kateryna Hrytsaienko
.png?w=240&auto=compress,format)
.png?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
Warmwind
Warmwind
LEONHARD WEISS GmbH & Co. KG