This is how it's done! Private, open-source AI models running locally.

Q: How much storage do the models take? (EDIT: 200MB according to the post - yeah, in this case, this better be a downloadable 'module' instead of being built-in) Could you make this feature optional, which would require the user to opt-in and download or delete the model(s) themselves? I don't want Firefox to go the Microsoft Edge route, where they shovel every feature under the sun, the user has no choice, and there is no way to reduce the storage occupied by the browser.

Looks interesting at first glance.

Thanks for being this open and transparent about the process, used model etc.

Nice work!!!

This would be particularly useful for postings to Mastodon, where alt-text is much socially desirable.

hm, i think this can be useful, however the problem is when people will never look at the output and just accept it at face value.

Basically I hope you will add a warning box that says "Do note that the text generation is not perfect and you should make sure the text clearly fits the image" or something along those lines. Also when it generates the text, it should always add "This alt text was generated by Firefox language model." as the first sentence, so people who rely on alt text features will know that this may be inaccurate.

would be amazing if this offered an API for webapps to use. E.g. mastodons alt text field could detect it has that feature available and provide a suggested alt text to users.

On the other hand this might encourage lower quality alt texts, as that will always be quicker to do than writing down your own alt text.

Maybe keeping it "fallback" only for consumers of content that is missing alt text is best.

no. get this garbage out of here

1. what the hell is it trained on? the same corpus of implicit bigotry and medical misinformation as every other "ai"?
2. it's just going to lie a bunch like every other "ai". wrong alt text is worse than no alt text.

What a shitty idea.

Wish I was surprised that Mozilla has jumped on the AI Highway to Hell.

But their priorities have not been user focused for years IMO.

youtube.com/watch?v=4hhlQU0zDp…

AC/DC - Highway to Hell (Live - from Countdown, 1979)

"Highway to Hell" from Countdown, 1979 by AC/DCListen to AC/DC: https://ACDC.lnk.to/listen_YDSubscribe to the official AC/DC YouTube channel: https://ACDC.ln...

^YouTube

Automated alt-text isn't too bad. This is a fair use of AI that doesn't really step on any toes.

I could foresee this causing problems if the alt-text is very wrong, though.

"The first time the user adds an image, they’ll have to wait a bit for downloading the model (which can take up to a few minutes depending on your connection) but the subsequent uses will be much faster"

Hope you can disable this feature and 200 mb download at all
I understand it can be very useful for some users, but for me not at all
If I find porno images I don't understand I think I can skip them 🤪

I'm still waiting for the feature that obligatory speaks back the comment to the user before posting.

(see xkcd.com/481/)

Listen to Yourself

^xkcd

I don't actually Want that in my browser.

If I did, I suppose I'd run, I don't know, Fucking Windows?

@hipsterelectron @sasha92 There are 3.2B images uploaded in a day (1.2T /year), many of them are repeating, google has 130B indexed. You can't describe all of that. Sure, human description will probably be better in many cases, but AI descriptions are still very useful.

Also, I doubt that project will get as many people editing it as Wikipedia has, so it can be great for a few popular images, memes, etc. but it can never cover random images on social media and websites without alt text.

Sensitive content Click to open/close

#FuckAI

My photo collection search could use such a thing.

#digikam

@paper @sasha92 I want to say explicitly that I don't fully agree w/ @hipsterelectron here, but I think that your dismissal of their concerns also substantially misses the mark

I've also talked at length about fine details that assist the Blind folks @weirdwriter talks about in using AI above while not incurring discussed harm. Making it easy for a screen reader user to knowingly get an AI description is relatively simple and worthwhile, but giving that tool to writers needs great care

These modules need to be opt-in from the beginning ... not mandatory with a "maybe we'll let you get rid of them some day, teehee" that seems to be your current plan.

I REALLY don't want that functionality, and especially not the bloat or other resource use. Guess it's time to start looking for a new browser.

@0x5DA It can be useful for a person using a screen reader to have access to an AI description, but crucial there is that said user needs to know that that's the source of said description. There are repeated patterns of those who feel pressured to include descriptions but don't actually care about accessibility doing the absolute minimum, manifesting here as using the direct AI output without examination or editing.

So yes, a lack of inline alt text is better than AI gen inline.

> said user needs to know that that's the source of said description.

this is a browser feature that the end user turns on or off themselves so yes they do know that. it's not being done by the publisher.

@0x5DA I don't have capacity in this moment for the full depth, but it's a bit more complex. Writing alt text that's actually equalizing of access, especially on social media, requires knowledge of multiple layers of context in which an image appears. Similar to other AI types, AI description works impressively *sometimes* but falls down hard on many types of image commonly appearing on SM, often in ways not obviously bad to those writing alt text.

An example: bsky.app/profile/hannah.the-vo…

Hannah is probably online 🟢 (@hannah.the-void.social)

People celebrating AI image descriptions but critically examining other AI usages frequently fall into a similar trap without realizing it.

^{Bluesky Social}