Baybe what's wrong? You've barely touched your Strawberrumjlai.luimage ElCanut ( @ElCanut@jlai.lu ) solarpunk memes@slrpnk.net • 2 months ago message-square17fedilinkarrow-up1271
arrow-up1271imageBaybe what's wrong? You've barely touched your Strawberrumjlai.lu ElCanut ( @ElCanut@jlai.lu ) solarpunk memes@slrpnk.net • 2 months ago message-square17fedilink
minus-square Even_Adder ( @Even_Adder@lemmy.dbzer0.com ) linkfedilinkEnglish5•2 months agoIt can’t see what tokens it puts out, you would need additional passes on the output for it to get it right. It’s computationally expensive, so I’m pretty sure that didn’t happen here.
minus-square ramirezmike ( @ramirezmike@programming.dev ) linkfedilink1•2 months agodoesn’t it work literally by passing in everything it said to determine what the next word is?
minus-square adderaline ( @ondoyant@beehaw.org ) linkfedilinkEnglish1•2 months agoit chunks text up into tokens, so it isn’t processing the words as if they were composed from letters.
It can’t see what tokens it puts out, you would need additional passes on the output for it to get it right. It’s computationally expensive, so I’m pretty sure that didn’t happen here.
doesn’t it work literally by passing in everything it said to determine what the next word is?
it chunks text up into tokens, so it isn’t processing the words as if they were composed from letters.