Anthropic publicly releases AI tool that can take over the user’s mouse cursorarstechnica.comexternal-link Pete Hahnloser ( @Powderhorn@beehaw.org ) TechnologyEnglish • 5 months ago message-square6arrow-up141
arrow-up141external-linkAnthropic publicly releases AI tool that can take over the user’s mouse cursorarstechnica.com Pete Hahnloser ( @Powderhorn@beehaw.org ) TechnologyEnglish • 5 months ago message-square6
minus-square Kissaki ( @Kissaki@beehaw.org ) linkfedilinkEnglish6•5 months agoIIRC Windows has an accessibility feature where the cursor jumps to the primary default action in opening dialogs. Doing it screenshot based seems inefficient if y du could iterate through windows and controls.
minus-square flashgnash ( @flashgnash@lemm.ee ) linkfedilink3•5 months agoMakes it work universally, even if the gui isn’t made with a standard toolkit Also it’s ai they don’t care about efficiency
minus-square Toribor ( @Toribor@corndog.social ) linkfedilinkEnglish2•5 months agoYeah this is one of those things where accessibility settings can probably get you 90% there but screenshots and machine learning can probably close the gap somewhat reliably (even if it’s much less efficient).
IIRC Windows has an accessibility feature where the cursor jumps to the primary default action in opening dialogs.
Doing it screenshot based seems inefficient if y du could iterate through windows and controls.
Makes it work universally, even if the gui isn’t made with a standard toolkit
Also it’s ai they don’t care about efficiency
Yeah this is one of those things where accessibility settings can probably get you 90% there but screenshots and machine learning can probably close the gap somewhat reliably (even if it’s much less efficient).