@
Christian Kent 𝘊𝘒 :\﹥ I can't really go with that proposal.
First of all, if you turn the automatic generation of alt-text on by default, then fully-abled people with
tons of spoons who could write their own alt-texts with no problems will use that automatic alt-text-generating AI, too. Especially newbies who'll keep everything at default settings for quite a while. This would make alt-text quality and especially accuracy
plummet.
Trust me. I've pitted LLaVA against my own hand-written image descriptions twice. In both cases, the images were renderings from very obscure 3-D virtual worlds with a resolution of only 800x533. The results were utterly devastating by my standards. Granted, the AI only had the images whereas I described the images while looking around in-world, being able to move the camera and zoom in on things and such.
So instead of turning AI-generated alt-text on by default, people should be educated about alt-text and told to write them themselves. Only if they can't, they should be told about the alt-text generator switch.
Besides, having this established on Mastodon is one thing. Trying to force
the whole Fediverse to implement an image-describing AI is something else. And: It will never come to pass.
The Fediverse is far from being only Mastodon. The Fediverse is not even a) Mastodon, b) Mastodon forks and c) stuff (actually or seemingly) glued onto Mastodon as add-ons after the fact. There's stuff in the Fediverse that's vastly different from Mastodon while at the same time being in direct competition against Mastodon.
For example, you have rather minimalist stuff like snac2 or GoToSocial. I'm not sure if their devs are too keen on including an AI-based automatic alt-text generator even if the AI is external.
On the other end of the spectrum, you have Friendica, Hubzilla, (streams) and Forte. These four are vastly more powerful than Hubzilla by various degrees. And they're so modular that, in theory, it'd be possible to build an add-on that uses an AI to automatically describe images.
Now, the first issue is that neither of these four strives to be more like Mastodon. In fact, they've got their own cultures that date back to times when Mastodon wasn't even an idea. After all, Friendica is five and a half years older than Mastodon, and the others are part of the same big family tree of forks and forks of forks. And, unlike Mastodon, they're Facebook alternatives with a side of fully-featured long-form blogging and not minimalist Twitter alternatives (and Hubzilla is much much more than that even).
Their cultures are vastly different from Mastodon's and will always be. They don't aim for brevity, what with "character limits" with eight digits. They don't ask for CWs in a dedicated CW field. What's the CW field on Mastodon has been the abstract field on Friendica for almost seven years longer and on StatusNet for some nine years longer, and like Friendica, the other three use it for summaries. All four have always been able to automatically generate CWs individually for each reader, depending on a) the presence of keywords in a message and b) the keywords in that reader's "NSFW" keyword list. Something that Mastodon didn't have until October, 2022 when Mastodon 4 came out, and that is not part of Mastodon's culture because it came that late.
As for alt-text, users on all four are completely unaware of its existence at best. That's because they're completely out of touch with Mastodon and its culture. In fact, some Hubzilla users chose to leave ActivityPub switched off, and some (streams) users decided to turn it off themselves, in both cases with the explicit intent of locking Mastodon out. Of those few who have heard about alt-text in the Fediverse, most see it as just another stupid Mastodon fad that Mastodon is trying hard to force upon the rest of the Fediverse. Like that 500-character limit.
Also, especially the Hubzilla devs and the (streams) and Forte dev are not too keen on implementing Mastodon features to say the least. For now, they only do what's absolutely necessary to federate with Mastodon, but not much beyond that. These three will not make an image-describing AI add-on a priority task. Or a task at all. And don't ask for building that image-describing feature into the core. This will happen even less. Also, don't expect either of them to hard-code a connection to a power-hungry corporation like Alphabet (Google) or OpenAI into an add-on, much less into the core. Despite the fact that Hubzilla still has its old (optional, off-by-default for both hubs and channels) Twitter connector.
By the way: (streams) and Forte are the only server applications in the Fediverse with a server-wide user agent filter that's not only capable of blocking Mastodon in its entirety, but that was actually announced and advertised as having that capability while having actually been designed to keep Threads out. This should give you to think.
There's another obstacle: Where would the alt-text be stored? After all, the handling of images is, again, vastly different on these four than on Mastodon. On Mastodon, images are files attached to posts with their own alt-text data fields. And upon uploading the images, they end up in some data nirvana.
In stark contrast, Friendica, Hubzilla, (streams) and Forte all have a cloud file storage built into each account (Friendica)/channel (Hubzilla, (streams), Forte), complete with its own simple file manager and, except for Friendica, even WebDAV connectivity. It's there where image files are uploaded. And then they aren't attached to posts as files. Rather, they're embedded into posts just like into webpages or blog posts. You can have text, then an image, then more text, than another image, then more text. These four can generate something that Mastodon still staunchly refuses to even display.
Now, I know that (streams) and Forte do have a field for an image description. It can be filled out during upload or in the image editor after the upload. I'm not sure if Friendica has at least got a data field for alt-text. But I know for a fact from daily-driving it that Hubzilla does not have any data field for alt-text, not in the UI and most likely not in the database either.
In the posts, the alt-text is not added to the images by referencing that data field. The posts don't work on some murky hocus-pocus in the background. By default, they're composed in a markdown language in plain sight. Friendica uses expanded BBcode or optionally Markdown. Hubzilla uses even more expanded BBcode. (streams) and Forte can use the same expanded BBcode as Hubzilla and Markdown and HTML within the same post. And the alt-text is always part of the image-embedding code.
And this is also the only place on Hubzilla where alt-text can come into play. This means that if Hubzilla wanted to include automatic alt-text generation, the alt-text would have to a) be generated whenever an image is embedded into a post while composing the post (= you embed the same image file a hundred times, and a hundred times will the AI generate a new alt-text for it from scratch) and then b) be woven into the image-embedding BBcode.
Speaking of image storage, WriteFreely, a specialised long-form blogging server application and basically a Medium alternative, doesn't even have that. You can embed images into posts on WriteFreely, but only as long as you can store them someplace that allows for hotlinking because you will have to hotlink your images from there. But if WriteFreely doesn't handle images itself at all, if images in WriteFreely posts don't actually go through WriteFreely until
after the post is sent (namely when someone views the post for the first time), it can't use an AI to automatically describe them either.
Unlike WriteFreely, Plume does have its own dedicated image storage. However, while Plume is not dead, its development has come to a complete halt for the foreseeable future because its devs don't have any time to work on it. And even if they got back to it, they'd probably have more important things to work on after several years of neglect.
And then there is software that is actually dead, that was officially pronounced dead by its own last maintainers, but that still has running servers. Calckey. Firefish. /kbin. But if it's dead, it won't even get security patches anymore, much less new features such as automatic AI-driven alt-text generation.
CC: @
James Edwards @
Danny Boling 
#
Long #
LongPost #
CWLong #
CWLongPost #
FediMeta #
FediverseMeta #
CWFediMeta #
CWFediverseMeta #
AltText #
AltTextMeta #
CWAltTextMeta #
ImageDescription #
ImageDescriptions #
ImageDescriptionMeta #
CWImageDescriptionMeta #
AIVsHuman #
HumanVsAI #
CW #
CWs #
CWMeta #
ContentWarning #
ContentWarnings #
ContentWarningMeta #
Fediverse #
MastodonCulture #
FediverseCulture #
snac #
snac2 #
GoToSocial #
Friendica #
Hubzilla #
Streams #
(streams) #
Forte #
Plume #
WriteFreely #
Calckey #
Firefish #
/kbin