Q/A con DistilBERT
DistilBERT è una versione più snella e veloce del classico modello Bert Bidirectional Encoder Representations from Transformers e qui lo si propone nella versione per Question-Answer con modello base pretrained in lingua inglese.
!pip install transformers

from transformers import DistilBertTokenizerFast
from transformers import DistilBertForQuestionAnswering
import torch
import pandas as pd
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased',return_token_type_ids = True)
model = DistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased-distilled-squad')
from google.colab import drive
drive.mount('/content/drive/')
filename = "/content/drive/My Drive/crypto/backpropagation.txt"
with open(filename, 'r') as f:
context = f.read()
context
Backpropagation is a backward propagation of errors and is a powerful tool of the deep learning. With the Gradient Descent the backpropagation reduce the cost function and the time of execution. We now talk about of calculate the Gradient Descent. With the Gradient Descent we want find the weights that minimize the errors, the cost function, through some iterations for search the minimum. There are some method, we see the principal, the Batch Gradient Descent, the Stochastic Gradient Descent and Mini-Batch Gradient Descent. The first, the Batch Gradient Descent, is a deterministic method that start always with the same data and produce the same outcome. It calculate the cost function of all the input data and then update the weights through the Backpropagation. This process is very expensive in time and resources for load all data in memory, especially as data gets big, for found the best cost function. The Stochastic Gradient Descent, shortened SGD, is a stochastic method.
domanda = 'what is Backpropagation ?'
tokens = tokenizer.encode_plus(domanda, context)
input_ids, attention_mask = tokens["input_ids"], tokens["attention_mask"]
startRis, endRis = model(torch.tensor([input_ids]), attention_mask=torch.tensor([attention_mask]))
risToken = input_ids[torch.argmax(startRis) : torch.argmax(endRis)+1]
answer_tokens = tokenizer.convert_ids_to_tokens(risToken , skip_special_tokens=True)
['backward', 'propagation', 'of', 'errors']
all_tokens = tokenizer.convert_ids_to_tokens(input_ids)
Predicted Answer = tokenizer.convert_tokens_to_string(answer_tokens)
Input Answer:
what is Backpropagation ?

Predicted Answer :
backward propagation of errors