Anthropic publicly releases AI tool that can take over the user’s mouse cursorarstechnica.comexternal-link Pete Hahnloser ( @Powderhorn@beehaw.org ) TechnologyEnglish • 30 days ago message-square6arrow-up140
arrow-up140external-linkAnthropic publicly releases AI tool that can take over the user’s mouse cursorarstechnica.com Pete Hahnloser ( @Powderhorn@beehaw.org ) TechnologyEnglish • 30 days ago message-square6
minus-square Kissaki ( @Kissaki@beehaw.org ) linkfedilinkEnglish6•29 days agoIIRC Windows has an accessibility feature where the cursor jumps to the primary default action in opening dialogs. Doing it screenshot based seems inefficient if y du could iterate through windows and controls.
minus-square flashgnash ( @flashgnash@lemm.ee ) linkfedilink3•29 days agoMakes it work universally, even if the gui isn’t made with a standard toolkit Also it’s ai they don’t care about efficiency
minus-square Toribor ( @Toribor@corndog.social ) linkfedilinkEnglish2•29 days agoYeah this is one of those things where accessibility settings can probably get you 90% there but screenshots and machine learning can probably close the gap somewhat reliably (even if it’s much less efficient).
IIRC Windows has an accessibility feature where the cursor jumps to the primary default action in opening dialogs.
Doing it screenshot based seems inefficient if y du could iterate through windows and controls.
Makes it work universally, even if the gui isn’t made with a standard toolkit
Also it’s ai they don’t care about efficiency
Yeah this is one of those things where accessibility settings can probably get you 90% there but screenshots and machine learning can probably close the gap somewhat reliably (even if it’s much less efficient).