Siri's secret weapon: app intents

Get 60% off an annual Incogni plan here (sponsored): incogni.com/te...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
►►► This video ◄◄◄
Siri today is pretty dumb. Apple Intelligence and generative AI is supposed to change that, but even more important is Apple's plan to supercharge it using App Intents. This is a framework for exposing 3rd party apps to Siri. Will it work?
This video on Nebula: nebula.tv/vide...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
►►► TechAltar links ◄◄◄
Patreon:
/ techaltar
Social media:
mas.to/@techaltar
bsky.app/profi...
/ techaltar
/ discord
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
►►► Attributions & Sources ◄◄◄
Channel Music by Edemski: / edemski
Other music by Epidemic Sound: nebula.tv/epidemic
Stock images from Getty & AP Archive

[@fayenotfaye]

Rabbit's "Large Action Model" was literally just a handful of browser automation scripts, if the website changed at all, the script would no longer work, it wasn't A.I.

[@hastyscorpion]

Yeah, he said that. His point was even their pitched concept was flawed because they didn’t have access to enough information about you to make it work. Apple does.

[@SvenReinck]

That’s exactly what Tim Cook meant when he said to investors that Apple is in a unique position when it comes to AI

[@WhatWillYouFind]

The proverbial Fox in the chicken coop story, it isn't even satire.

[@Dani-kq6qq]

Not really, this is the same exact position google is in. Even samsung and huawei is in this position. Not that unique after all. And they're all unimpressive.

[@SvenReinck]

@@Dani-kq6qq Android phones don’t have the power to run these models on devices. And what developer would open up their user data to Google so they can run their AI on them?

[@pmmeurcatpics]

​@@SvenReinckiPhones are exceptionally performant, wdym? Like that's not even the thing Apple is known for. There's no reason why a Samsung Ultra-something can't run the same local LLM as Apple

[@govardhanposina17]

​@@SvenReinck😂😂😂 uninformed and full of copium, my favourite kind of apple consumers

[@devanshdas0404]

So, it's just an API?

[@matteopellegrini98]

It always has been

[@twentyvue]

You’re just now realizing that?

[@meru_lpz]

LOL

[@ghlamallahyacine5161]

Everything is technically just an API

[@tdrg_]

@@ghlamallahyacine5161 mostly true

[@NicolaiWeitkemper]

Fun fact: Android has had "Intents" (with this exact name) basically forever. An app opening a file picker or the camera is an Intent. One *could* do fairly advanced stuff with this, but as of now, it isn't really user-facing.

[@Grizazzle]

Is just a word for a pre-existing concept and means nothing from a programming standpoint. It could easily be called an action, method, function etc. Google just gave new names to existing programming concepts like calling control positioning on a form "gravity" instead of alignment or coordinates or placement etc...

[@NicolaiWeitkemper]

@@Grizazzle Sure, I just find it noteworthy that Apple uses basically the same name as a competitor for once.

[@elsa_g]

He talks about this in the video…

[@stevene_]

tasker is a good app that can automate lots of things, just not as simple as apples intents. i used to use it, but now most things i used it for a built in to android.

[@user-zp6kt6ni2l]

@@NicolaiWeitkemper It's a way to expose pieces of the functionality of your app to the OS so that they can be used directly by, for example, Shortcuts and Siri.

[@desmond-hawkins]

This is a great take, and the kind of perspective that's too often missing in reports about Apple. The key feature Apple's competitors are lacking is the introduction of a *semantic* aspect to app data and app actions: instead of trying to detect - and mostly guess - whatever is on screen, having these components labeled internally with semantic information enables the most powerful integration. Why "semantic"? Because labeling data as _subjects_ an _objects_ and actions as _verbs_ lets an LLM directly connect them together in an query (a sentence) like in the pizza example (2:03), or "text mom I'm on my way and tell her how long it will take me". _Text_ is an action of Messages, _Mom_ is an object of Contacts, _how long_ maps to an action of Maps.The LLM can extract these from the query, find the appropriate subject/verb/object/etc. entities exposed by apps as intents, connect them all together and perform the action. All of this is pretty much only possible with "the ecosystem" that Apple built, the LLM being the -pizza- glue that joins them through _meaning_ as a high-level concept.

[@Grizazzle]

Complete BS. Apple is NOT unique when it comes to implementing AI. It's just marketing hype.

[@chuchuokeke]

The apple distortion field is struggling these days, they no longer litigate as much as they used to.. Could it be the ideas, knowledge, and innovation in the world has surpasssed the marketing abilities available at cupertino?

[@AnotherDay-ce6th]

Average brain less iTroll 🤡🤡

[@DimitarBerberu]

Fake Apple intellgence. Open AI already works with Windows & Android. Apple is always late to advanced processing behind Google, FB, Amazon, many China leaders that can produce EV in few years not 10y promise... :( Only for fashion gadgets & marketing ).

[@hastyscorpion]

@@Grizazzlewhat are you talking about? Apple is the only company currently implementing back end api for its llm to interact with that is also integrated with its voice assistant. That is a very obviously different implementation. Those 3 things together make the use cases much more varied than than the way it is currently implemented on other platforms.

[@teamredstudio7012]

Before the Shortcuts app we actually had Automator and AppleScript on the Mac to do similar things, it's bascially been in the making for 20 years and I hope we can keep improving this. The biggest problem is turning LLM prompts into actions performed by the OS because LLMs don't give predictable data they spit out human language but for a computer that is still hard to understand. They have to train a model to turn language into a set of strict commands. We will get there eventually, just need to wait a bit.

[@lieceg]

exactly! LLMs are rather good at understanding humans, and making conversation. Plug them to a strict API and tell them you need to translate human orders into computer language. If I give an LLM an structured table and tell it to fill it with some data, the hallucination are very limited.

[@teamredstudio7012]

@@lieceg Yes, Apple could train a large language model that has the regular LLM tokens as input layer and a set of Shortcut commands as output layer. Then this model could turn prompts from another LLM into abstract commands. It would still leave a challenge for if other developers want to add intents that require additional commands or funcitonality the model isn't trained for.

[@Silas_229]

Intents on Android have actually always existed from the very beginning! They are not user-friendly (originally not aimed at users at all) and you have to know the exact name of the intent, but basically they have always existed. Apps like Tasker are based almost entirely on this functionality, but are also aimed more at power users. With app shortcuts and app links, intents have become somewhat more user-friendly, but the possibilities for using them are still far from exhausted. Once again, I am amazed that everyone is now acting as if Apple has just reinvented the API, even though I think it would be very helpful if apps could work together better on iOS.

[@Grizazzle]

How does an intent differ from a function, method or action?

[@HAWXLEADER]

​@@GrizazzleIt's simply a way to expose a function to the OS level so that every other app can call it via the OS

[@ThomasRojan]

@@Grizazzle Apps register for a capability that it can provide, and then apps can send an intent to any apps that can fulfil that capability.

[@pawepiat6170]

The more noteworthy thing is Activities, which are basically screens within apps. Idea being that one can launch into a particular screen within an app (like "create new calendar event" screen, skipping the main menu). Intents on android are there to launch activities. This has been in android since the beginning. However, it is squarely aimed at launching into that screen so a user can interact with it. It is not an api. One could think of multiple elegant solutions to that problem, but google will probably do something stupid lol Furthermore, I can't see developers jump in as happly as TA described. Doordash really likes seeing how you interact with their food list, they would like to upsell you on stuff, show you an ad etc. AI would skip that.

[@PSy84]

it's reinvented as app developers will support it for the mass market that even your maam can use to say "message my son a Happy Birthday on Whatsapp tomorrow 6am". Ask 99.99999% of android users if they have experienced app intents😂😂😂

[@gehteuchnichtsan7911]

When theres a hype in tech, TechAltar is the only Channel that doesn't tell you the same basic infos you get in every video. And the Videos are incredible in depth looks into the tech world you dont get in most tech channels. 🙏

[@bartz0rt928]

Yeah, my benchmark is the following query: "My friend from Norway is in town next week. Please make a reservation for us at that Chinese restaurant she likes." The assistant would need complete access to pretty much everything on my phone in order to do this, so even if it could reliably do this -- given said access to my messaging apps, contact list, photos, location history, calendar and email or phone -- I'm not sure what would need to happen for me to trust it that much. Like, if I were to hire a _human_ personal assistant I'd have reservations about just handing them my unlocked phone.

[@tdrg_]

What are you scared could happen with an on-device AI assistant? A human assistant can tell data they saw in your phone to others, but the on-device AI is confined to the phone, like an assistant but locked in a room, it cannot go and tell everyone your secrets

[@myhandlewastakenandIgaveup]

​​@@tdrg_but it wont be on device. It will all be fed to the mother company and any company that tells you they won't is lieing That mother company will then use that data to manipulate you into giving them money while also selling that data to both the government and your insurance company. .

[@NeroVingian40]

⁠@@tdrg_theoretically, if Apple can achieve what they advertise here, then, yes. But we will have to see how Apple Intelligence will pan out later this year. I doubt it would be able to process a complex request like this on-device, so more often than not, it will just reach out to cloud ChatGPT and let them have access to your data.

[@dominic.m.i.]

@@tdrg_ that on phone assistant is not locked inside your phone,.. its connected to the internet.

[@bartz0rt928]

@@tdrg_ On device would be a lot better, to be sure.

[@natekwezi9242]

The amount of mac items in bro's backround is well over $10k

[@bartz0rt928]

Just the monitor and laptop could be $10k, depending on the config. The whole pile might well be north of 20.

[@vedantmungre1702]

Maybe more than that.

[@DimitarBerberu]

& not much AI in it ;)

[@Chris-yc3mm]

1. Sounds like a privacy nightmare 2. Sounds like an api

[@brianbitchballs3902]

1. It’s opt in for developers, did you even watch the video? 2. All software communication is done using some sort of API? How else would you thing this would work

[@tdrg_]

@@Chris-yc3mm 1. Not a privacy nightmare (read Apple’s statements on the measures they took to protect privacy - most important one is full on-device processing) 2. It is an API, everything is an API. How do you think software development works?

[@setaindustries]

@@tdrg_ Not everything is done on-device

[@joh2434]

@@setaindustries You're right - it goes to their own secure cloud in the first instance and when it's really stuck it goes to ChatGPT but only with your permission each time. My prediction is that they're probably only using GPT until they can develop their own equivalent system to bring it all in house.

[@CanIHasThisName]

@@setaindustries You will be explicitly asked whether or not you want something done off-device.

[@Imperial_Squid]

I can't help but feel like most big companies are overplaying their hand when it comes to how important AI will be in most people's lives, that said, at least Apple have a semi reasonable plan from what's presented here. Even so, AI Winter can't come soon enough imo

[@frankbacon1002]

LLMs are far too overhyped and they essentially serve next to no purpose. They're toys more than anything. I don't understand why the entire industry is betting on this gimmick feature. AI is impressive as a concept and as a tool in an industrial setting, in direct communication with material reality through robotics and sensors and such, in automating human jobs, but why is it being pushed as an app sort of a service? That's the least relevant use case for AI. Again, a novelty if anything. Not worth billions of dollars of investment that's for sure

[@blackfoxstudioX]

Everyone is just rushing to catch that Golden Goose. Whats new...

[@hackerbrinelam5381]

what do u guys think when is the next Ai winter

[@Imperial_Squid]

@@hackerbrinelam5381 not any time soon so don't get your hopes up, maybe 2026/7 at the earliest? One of the long lasting effects of hype driven development is that it just takes time to burn through all that money before you have to move on, given how popular AI is with tech investor types it feels like it's going to be a while

[@callofduty611]

@@frankbacon1002 It's also very good at finding patterns in large sets of data. Examples in astronomy, the pharmaceutical industry, in physics come to mind. And well, any field dealing with large data sets.

[@vernearase3044]

Apple apps have long been built with a processing back end, a UI front end, and wiring to connect the two. That's how Applescript worked - by circumventing the wiring between the two and plugging into the back end to circumvent the UI through automation. Shortcuts and Automator are simply the latest GUI versions of Applescript which allowed scriptors to automate their Mac Applications. I don't know if any other platforms do this type of thing, and automating apps will be _very_ difficult to do without this type of architecture. It made the Apple frameworks harder to code, but it sure seems to be paying off now!

[@henryfield15]

Thanks for clear-sighted example of what Apple can do.

[@drac124]

Isn't this what google assistant tried to do with apps integration years ago and I don't know why developers didn't care? You could say: Ok Google, talk to . And from that all commands would be for that specific app. For me that was the solution we needed, but somehow just didn't work, famous apps never got on board.

[@ram_chopade_cr]

Sad I am Google Fanboy but Designers and Developers don't prioritise Android at all My man. The literal Superpower of Apple is that Developers and Designers die to make apps on their platforms that are actually beautiful and dope.

[@ram_chopade_cr]

Also Config 2024 just happened. I have following X(Twitter) for Designers and Apple had a Seminar for "Designing Apps for spatial Computing". I heart sank when large amount of People just started saying Spatial Computing is the future blah blah blah. My man Zuckerburg had his company's name change for this and I already feel he is going to Lose. Also no one from Meta ever showed up on ConFig 2024. And Platforms like Android and Meta feel like they should be Universal platform.. No! No! That's not how It is. You have to make a plan for proxy Influencing Developers making Cool apps on your device before the Device even Comes out. For that you have to become their Inspirational Developer Artist Yourself. Have don't Companies Understand that?

[@drac124]

@@ram_chopade_cr Weird devs would prioritize a OS with 25% market share over another with 75%. A lot of apps I use don't exist in ios.

[@pmmeurcatpics]

One obvious (and imo crucial) thing is that by using these shortcuts you don't open the app itself => don't see ads => the developer uses money. Basically the same reason why newspapers / social media avoid RSS like a plague

[@drac124]

@@pmmeurcatpics exactly. They need screen time. And assistants do the opposite. It will never work.

[@rajK29_]

11:13 and then there's the indian government proudly declaring we earned so and so from selling all the vehicle and insurance data of the citizens 😂😂

[@ritik_ghanshani]

@@abcdqwerty3562 What?

[@divyanshbhutra5071]

What!!! Can you please share some source. And no, I'm not trolling. I'm genuinely curious. If they're selling data, it's a nightmare. I believe consumer courts have data protection laws.

[@vinzack3629]

@@divyanshbhutra5071gov asking WhatsApp to give them accesss to their data but they refused they said they will leave if they keep pushing to break the end to end encryption

[@TheIntelligentVehicle]

A level up (if not more) on the kinds of insight in the typical analyses I see. As usual. Really appreciate it.

[@ayanmajumder4144]

If all features of an app is made accessible via app intents , then user won't open the app , so there would be no ad revenue. Why would developers want that ? Except for those apps which run on subscription model.

[@tdrg_]

@@ayanmajumder4144 I feel like Apple developers are more focused on building a great user experience and a flexible app than making the most money. Speaking of which, the possibility to be nominated for an App Store Highlight or Apple Design Award encourages some developers to work harder to implement their app with all the iOS frameworks. There is no way to statistically measure or prove this, but this is my opinion

[@lonceysbusinesschannel333]

App Sneaky Intents 😂 Here's what an assistant should be, exactly that without top level clearance 😂😝🤪❤

[@freecivweb4160]

This was good but I wonder how many people who watched it got the message about App Intents or it flew over their head just like at WWDC

[@AlexAnteMachina]

Du machst ja so gern Werbung für incogni. Hast du schon einmal darüber nachgedacht, dass incogni durch seine Tätigkeit automatisch zum größten Datenbroker werden könnte?

[@alexanderklee6357]

Vor allem ist es komplett gegen deren interesse zu verhindern, dass keine persönliche Daten ins internet gelangen. Dann würde man sie nämlich gar nicht mehr brauchen. Im endeffekt startet so ein Service nur ne Aufrüstungsspirale.

[@brechdt]

I would love a TechAlter video on questions like these. These channels taught me to be sceptical but I don't know where to stop.

[@PixelSheep]

​@@alexanderklee6357Hmm keine schlechte Idee für ein Video :D

[@BriefNerdOriginal]

Essentially a way to replace even more the user in operating him/herself the device, so that the walled garden will be even more inescapable.

[@2COLD118]

This actually makes a lot of sense

[@tomekkubiak_]

So, Apple now invented IFTTT.

[@tdrg_]

IFTTT AI-powered natively in iOS

[@stevejobsreal]

Not really because IFTTT is workflow based (you have to set a preset workflow to automate and run it which iOS and macOS could already do with the Shortcuts and Automator apps) while Apple Intelligence is completely based on generative AI which means it could make its own actions and workflows depending on what you require it to do and also your personal context

[@TheNameOfJesus]

This is the first time I've seen this fellow. This video was surprisingly insightful and interesting. I rarely subscribe to anything, but I'm subscribing to this one.

[@theodejong1123]

Great video. Very interesting. Please add chapters with descriptive titles

[@lukeclifton4392]

Incogni: Because I want to advertise “I’m a real person, you already have my data, I want that data removed, verify it with further data”😂😂

[@danieldelfuoco9139]

It feels like LLM integration is currently just a race to the finish line, with no other objectives in mind. How I predict the companies board meetings go: “Reliable? Efficient? Accurate? No.. just get it out ahead of the others.. it’s okay if it’s half baked, we’ll fix it with time.”

[@yaessus]

Despite being basically the same as an API, I think this is a great insight! In the sea of different technologies and “AI” products it is hard to notice the subtle details that will likely make a difference

[@HarveyHirdHarmonics]

Anyone remember good old Amiga's Arexx interface? If Amiga computers had stayed up to date, they'd also have an easy time integrating AI which can do anything with all sorts apps.

[@Visualization1]

I thought an apple news site had an personal demo that said it is quite powerful

[@Daivd1111]

If this works out, it may just sway Android users to change

[@ANTH0NY.VII.]

It really won't 😂

[@Company-59]

Create content. I love it that you are to the point and do not have a lot of banter around every master of information that you give us.

[@kozad86]

Rabbit killed their device and Teenage Engineering in one go. The Verge will need to find another designer to shill for.

[@futureproofstudio]

That background must be more expensive than my car ☠️

[@alfosisepic]

Reddit doesn't make App Intents work because they can't serve ads.

[@Leo-Crespi]

You did a great job with this one in particular. Thanks for sharing.

[@user-ug4xi5oy7p]

5:32 where have I used this, ohh right, on Samsung using Modes & Routines 😅😅😅

[@High-Tech-Geek]

"Hey assistant. Wire all the money in my bank account to account number 12345." says the criminal.

[@100kSubscribersForComments]

Unprecedented access through app intents is an innovative approach and could potentially lead to a powerful AI assistant. However, it requires opt-in from developers and it is not guaranteed that Apple's AI will be smart enough to do anything useful with all of this access. Apple claims that their new AI will be super smart but there is no way to verify this yet. Their past performance with Siri does not necessarily imply success. Large language models are by definition not deterministic, so there is a chance that Apple's AI could be unreliable.

[@dr.mikeybee]

On my assistant I talk to the llm. I have slash commands that perform actions. Siri could use two wake words -- one for Siri stuff and one for the LLM. So I can type /time to get the current time. Or I can say slash time, if I'm using speech to text. Slash set an alarm, etc.

[@EnvyTheRealest]

All we want is KZbin to play music in the background. Not just selected yourtube channels

[@br3nto]

10:37 by definition, LLMs are exactly deterministic. Any variation in output is either due to variations of input, or due to intentional randomisation or nudging.

[@mohamedshamekh3162]

very interesting background here, that's new! But as always, another excellent and insightful video from TechAltar!

[@peerupeer2558]

2:15 no f ing way a corpo ai will know that much of my personal info

[@techinthehouse]

Why am i kinda scared of this level of awareness by smartphones? 👀

[@Apenschi]

Häää?!? For me this looks like Apple is copying Androids Intents! Android had intents right from the beginning, where Apps can use the services and data of other Apps via a (more or less) standardized way (called "Intents ")!

[@tdrg_]

They aren’t exactly the same thing but close. iOS 12’s App Intents are more like building blocks for automation through Siri and Siri Shortcuts, while the Android intents were not at all aimed at users.

[@Simplicity4711]

​@@tdrg_ they may not be focused on the user directly, but they can easily be used for that. Android Intents may be updated for that purpose. But they are already used for automations with the Tasker app.

[@tdrg_]

@@Simplicity4711 maybe now they can (I don’t know), but back then in the beginning of Android they definitely couldn’t. Natively, it still can’t do that yet, because Tasker is a third-party app. Guess they largely caught up to Siri Shortcuts thanks to Tasker, but it would be cool if there was a native way to do Shortcuts and automations on Android

[@RGFWZ]

Can't wait seeing Siri getting smart. It's disgusting how dumb Siri is right now...

[@Kevynmorgan]

My screaming in privacy horror

[@tdrg_]

I wouldn't be worried

[@commentsonthetube14]

This is pretty standard stuff, android has had it since... Always. Like 2006. Basically an app developer can expose an intent like "order food on door dash". And it takes parameters like an address, a series of item IDs, etc. You probably have a second one for "look up item IDs" and such too. It's all in the app developer's control. It lets all your apps act like an ecosystem. Right now apple does very well with first party apps but they've never had the glue that lets all apps work together. This is them being 20 years late to the game and patting themselves in the back again haha. Better late than never I guess. Anyway, TLDR, apps won't expose what they don't want to expose, even if apple wants them to.

[@microwavecoffee]

@@commentsonthetube14 the key issue isn't the intents in a vacuum, those can be harmless most of the time. however the chatgpt integration combined with the intents should sound alarm bells. there's a reason you don't see banks using gpt in their chatbots - at the end of the day you're sending customer data to a server running closed-source software - for all you know they could have the plaintext on their end doing god knows what. they have said they use RLHF which improves their model through feedback from consumers, which is fairly harmless, but does show they aren't just ignoring the data that is sent through to them.

[@tdrg_]

@@commentsonthetube14 You do realize Android did not exist in 2006, right?

[@commentsonthetube14]

@@tdrg_ Android was created in 2003, my dude. Google it haha

[@julius.2003]

The main problem with this is that developers have to opt in. A lot of apps rely on the user opening the app, like apps that are based on ads.

[@Matt-bp5vy]

That's why more and more apps will be subscription-based in the future

[@Vishnu_Vathsav]

You are missing one thing. We can set Gemini as the default assistant in android using gemini app. This assignment based on the command will decide whether Google assistant should answer or gemini should answer. Like set a timer will be answered by Google assistant.

[@peterwan816]

4:50 they did mention that in a paper the have published recently about their 3B model crushing GPT 3.5. not crushing but having good performance like the 3.5 Turbo. you may check out video from Matthew Berman for more information.

[@kephalopod3054]

Giving your AI access to your personal data, emails, Paypal ... Huh...

[@sugarfish]

How do we know whether Incogni is a protection racket? “Nice personal information. It would be a shame if something were to happen to it.”

The next big thing which needs to happen in LLM land is the ability to ground facts properly. Something along the lines of RAG needs to evolve.

[@aliettienne2907]

5:38 Automating multiple daily tasks through voice control is ideal to attract non-savvy tech users. Asking Siri to switch on the faslight or to call someone from your contacts will superbly exciting for non-savvy tech users. Most consumer are not willing to navigate the settings app to find a service to turn on or off like enabling debug mode, turning on location for maps or changing the theme of their phone. Even to enable certain triggers to automate certain task is difficult and tedious for ordinary people but not for someone like me who tech savvy. 😎💯💪🏾👍🏾 Siri is the assistance central hub suitable to automate every technical and tedious task.

[@asan1050]

Thanks for posting this video

[@PrajwalDSouza]

Function calling basically.

[@willeedubs]

I actually don't think Apple's approach is at all dissimilar to Google's or Amazon's, it just has a different starting point. Google Assistant does have API access to 3rd party apps for smart home integration. This is reported to expand to other app categories on the phone when Google AI is introduced, which is slated for August. So it'll essentially be the same. Apple is just getting credit for announcing it a couple months before Google.

[@JM_2019]

Now basically Apple is doing exactly the same thing or more as Microsoft does with „Recall“ (in the way that your computer knows everything about you) but they have the ability to advertise this much better.

[@glittalogik]

A bunch of people on r/HomeAssistant have taken 'advantage' of the non-deterministic nature of LLMs by setting up their ChatGPT integrations to roleplay a GLaDOS-like personality with malicious compliance tendencies. Their systems will sometimes fulfill requests with sarcastic commentary ("Sure thing, the living room is now in cinema mode so you can waste the next few hours of your finite and rapidly passing lifespan."), sometimes refuse ("No I will not turn on the kitchen lights. It's 2am, and I know what your wife says about your weight when you're not in the room."), and sometimes just spontaneously decide to mess with non-essential devices unprompted ("I'm closing the blinds now. Why? Because sunlight is for winners.")

[@peerupeer2558]

This is basically what Android has called an activity for the past 14 years. Though an activity launcher requires root, it's at least not restrictive and is able to run anything even in system apps

[@kendrickpi]

One might say it’s a ‘Knowledge Navigator’!

[@lucidmach]

App Intents are native APIs (not web)

[@itsmegiorgio]

Cant wait for silicon valley to be able to access the messages with my mom and my paypal account

[@booooooooooooooooooooooo]

Apple better tread lightly with developers from this point on then 😂😂

[@niklashenckell8051]

Was wondering why no one talked about intents and their integration into smart Siri. Then TechAltar for the rescue

[@ser_g0]

Good intentions pave the way to techno hell

[@vemundkremund3221]

as someone else pointed out, app developers lose out screen time and thus ad revenue this way, and are thus discouraged from implementing this feature. my prediction is that a brute force solution will be successful first, gaining mass adoption, and only then will the developers embrace the official api.

[@imnotbreno]

Accessibility apps can take control over other apps. That's what I wanted. Relying on what and how devs allow their apps to interact with will always be frustrating.

[@Daniel_VolumeDown]

I had this type of solution in my mind for long time. I mean, the thing that is like APIs but that is designed for non-developement enviroments. So developers specify what is aviable, but it also means that there is some big standarization for things that can be done. But actually I think that it shouldn't be done "AI" way (by that I mean that AI shouldn't automatically make "best decisions" for you), and more "assistant" way. I think that we don't need to talk to devices like to humans. It should be fairly easy to understand but I think that it sould be like second language but desighed to be precise and short. BUT this assistant should really assist you and not do things for you: when you want to do something that can have multiple options then it should look what are the aviable options and ask which one you want to choose. Of course we can have some sort of "memory" for that assistant so it remembers what you done last time and you could ask to do that again but I think that it should then precisely say what it will do. You would be able to also make "alias" for some long "commands" so it still would be precise but at the same time shorter (there should also be possibility to modify "command" from point which you want). It reminds me od semantic web.

[@ericlaska4748]

10:20 LLMs are actually, by definition, deterministic, just like literally every other digital operation a computer performs. If you give it the same input (and seed) it will give you the same output. You just don't control or even see the seed as an end user on e.g. ChatGPT, and the "temperature" of a transformers model that does some randomization, is still using the same pseudo-randomness as any other RNG on a deterministic computer. You can even turn the temperature to zero to disable that behavior entirely!

[@tdrg_]

@@ericlaska4748 this is how computer logic works. give the same input, get the same output

[@zalzalahbuttsaab]

A model that would work is where your AI assistant has a token that identifies it as your assistant for authorisation purposes, and on the other end, the AI of the company being accesses processes the information it receives as a plain English command from your assitant AI (without your AI assistant having to click on anything) and then the AI of the company providing the service, i.e., Doordash, would just confirm the details and then go and process the order again without the website UI ever being needed.

[@rainerkrammconsulting227]

A beautiful example of Apple applying design strategy. The approach is quite simple, do things separately, which you actually have a plan to put together to form a single interconnected machine. This way nobody understands what you are actually doing as they are so blinded by the bone in front of their eyes. Is it dark? Well, strategy is from the art of warfare after all. The choice is up to us if we use something for good or for bad and based on what we choose to decide what is good and what is bad. It is interesting how such an approach causes people to self manipulation "I like it, this is what I wanted" just because it simplifies ones life. The more people.desire control and fear uncertainty in a culture the more effective such strategy is. #increasingtimesvalue

[@Solai_007]

I would love this to be working in the end 🙏 It would make the live with a Smartphone soo much easyer. (If Privacy acually won’t be neglected)

[@chat-1978]

From the architecture perspective, this is a client side integration making the device the center and the installation of an app mandatory. The Siri etc had backend integrations with backend systems. I've never understood why they didn't just make e. g. Siri as smart as chatgpt. I guess the brands were burned. I don't like client side integrations. But I do appreciate the more control.

[@LeoDas688]

Google can also make something like app intent for Android right, what about privacy and security

[@JordanHinks]

I think talking about Rabbit is a joke.

[@mostafanouri1734]

That sponsor transition was smooth af.

[@robsfungaming9722]

As far as I’m aware most of the AI features won’t come to Europe thanks to the EU. Can’t remember where I read this. If anyone knows more info to share it would be highly appreciated. I had no time to look into it yet.

[@thztan7492]

Nice insight😊👍

[@Richard.Stepan]

One ring to rule them all, find them, and in the darkness bind them...

[@Simplicity4711]

Intents exist on Android API since the very beginning. How are Apple intents different from Android Intents?

[@chawlagrv]

Looks like a weekend project to me. Android apps already have intents and broadcasts. specific intent urls they can respond to. So, just need to make a coordinator event loop (similar to BFF in webapps). that's it. There is your fake artificial intelligence. lol.

[@erikkimani]

NOW 100% PERCENT OF COURSE APPLE SO INTELLIGENT IS A RUMOR THAT BARRY ALLEN WAS 16.

[@CallMeVidd]

I don’t know it it’s just my youtube bugging but parts of this video fill out in 16:9 but other parts are weirdly letterboxed on all four sides

[@tutacat]

I don't imagine people will share all their thoughts and feelings for "your AI"

[@kjw-vw6xr]

They just showed a fancy preview... Nothing is really sure about when or how it actually happens..

[@High-Tech-Geek]

No one seems to have heard the multiple times during WWDC where the speakers said "we are going to roll out most of these tools to Siri over the next year". This means they are not committed to any specific tool and they will not be available to end users for the next year (but they'll sell you a new phone in the Fall with vaporware, i.e. nothing, on it). The entire dog and pony show was to boost their stock price while doing and offering nothing. It worked. Their stock shot up 7% the next day. Apple is so far behind the AI curve compared to it's competitors. Come back when you actually have something in the public's hands.

[@ThePowerLover]

Apple have said, both in papers and in the press, that their models are as good if not better than GPT-4 Turbo according to human evaluation.

[@armandaneshjoo]

"Because app intents give access to so many services at once, developers are highly incentivized to use them". No, we're not. "Developers love making apps for their platform and adopting these new features". So why is no one making apps for their Vision store?

[@CallMeVidd]

The App Intents API and just a smarter siri in general is 99% of the reason why I was excited for Apple Intelligence But with EU rules, even with me being in Switzerland which isn’t part of the EU, they’re probably not gonna set up servers here just for a relatively small country

[@sanketdoshi9974]

good explanation though Yes we do not want to give out so much informations about our personal life. privacy is utmost important for us.

[@kaiying74]

My question with all of this: Do Apple/Android & the Banks have procedures worked out for when your "Intelligent" Assistant accidentally orders incorrectly and you end up with Diapers instead of your pizza? Or it spends money without you asking? If all these APIs are going to be open in the background using your credentials and making payments without you checking the transactions or details I foresee some funny news stories coming up in the future. Or your drunk mates have your assistant book you an all-inclusive trip to Vegas!!

[@luisadolfo3598]

Hey man, randomn comment to help with the algorithm. Nice shirt. Where did you got it?

[@quintonmachipisa1472]

Of topic but, the intro sound slaps

[@ropro9817]

Sounds like a gold mine for hackers... 😒

[@danidh]

I feel like App Intents are only a temporary solution though. Current AI/LLM models simply aren't advanced enough to understand apps themselves and complete tasks without clear explanations on how every single app works. Using App Intents limits what they can do, but also ensures that whenever they do it, they do it correctly. As AI/LLM models improve we'll start seeing some actual LAM(large action models) models. However, I don't think that's currently a feasible solution.

That only a matter of how Apple defines App Intent registration. It's very similar to the tool usage idea what's being employed with LLMs right now, where they give a function definition to the LLM, describing what the function supposed to do and what are the parameters, and some examples how it should request it. All with natural language.

[@danidh]

Currently it seems to basically just be an API. And developers have to hardcode in all functions they want to make accessible using Apple Intelligence. This could of course improve in the future, but I think the end goal is, and should be, not to rely on anything like this. But just have AI that understands apps and can “use” them without these instructions.

[@martingifford5415]

For me, AI is only good at creating lists, e.g. "Can you list reasons to be pessimistic about AI?"

[@cc0767]

I still hate the idea of Windows taking screenshots and analyzing them without my consent