if you could pick a standard format for a purpose what would it be and why?
e.g. flac for lossless audio because…
(yes you can add new categories)
summary:
- photos .jxl
- open domain image data .exr
- videos .av1
- lossless audio .flac
- lossy audio .opus
- subtitles srt/ass
- fonts .otf
- container mkv (doesnt contain .jxl)
- plain text utf-8 (many also say markup but disagree on the implementation)
- documents .odt
- archive files (this one is causing a bloodbath so i picked randomly) .tar.zst
- configuration files toml
- typesetting typst
- interchange format .ora
- models .gltf / .glb
- daw session files .dawproject
- otdr measurement results .xml
- palordrolap ( @palordrolap@kbin.social ) 85•9 months ago
Just going to leave this xkcd comic here.
Yes, you already know what it is.
- dingleberry ( @dingleberry@discuss.tchncs.de ) 3•9 months ago
🪛
- raubarno ( @raubarno@lemmy.ml ) 77•9 months ago
Open Document Standard (.odt) for all documents. In all public institutions (it’s already a NATO standard for documents).
Because the Microsoft Word ones (.doc, .docx) are unusable outside the Microsoft Office ecosystem. I feel outraged every time I need to edit .docx file because it breaks the layout easily. And some older .doc files cannot even work with Microsoft Word.
Actually, IMHO, there should be some better alternative to .odt as well. Something more out of a declarative/scripted fashion like LaTeX but still WYSIWYG. LaTeX (and XeTeX, for my use cases) is too messy for me to work with, especially when a package is Byzantine. And it can be non-reproducible if I share/reuse the same document somewhere else.
Something has to be made with document files.
- schnurrito ( @schnurrito@discuss.tchncs.de ) 17•9 months ago
Markdown, asciidoc, restructuredtext are kinda like simple alternatives to LaTeX
- d_k_bo ( @d_k_bo@feddit.de ) 10•9 months ago
There is also https://github.com/typst/typst/
- monobot ( @monobot@lemmy.ml ) 16•9 months ago
It is unbelievable we do not have standard document format.
- DigitalJacobin ( @DigitalJacobin@lemmy.ml ) English12•9 months ago
What’s messed up is that, technically, we do. Originally, OpenDocument was the ISO standard document format. But then, baffling everyone, Microsoft got the ISO to also have
.docx
as an ISO standard. So now we have 2 competing document standards, the second of which is simply worse. - Fonzie! ( @lord_ryvan@ttrpg.network ) 1•9 months ago
That’s awful, we should design something that covers both use cases!
- megane-kun ( @megane_kun@lemm.ee ) 12•9 months ago
I was too young to use it in any serious context, but I kinda dig how WordPerfect does formatting. It is hidden by default, but you can show them and manipulate them as needed.
It might already be a thing, but I am imagining a LaTeX-based standard for document formatting would do well with a WYSIWYG editor that would hide the complexity by default, but is available for those who need to manipulate it.
- raubarno ( @raubarno@lemmy.ml ) 8•9 months ago
There are programs (LyX, TexMacs) that implement WYSIWYG for LaTeX, TexMacs is exceptionally good. I don’t know about the standards, though.
Another problem with LaTeX and most of the other document formats is that they are so bloated and depend on many other tasks that it is hardly possible to embed the tool into a larger document. That’s a bit of criticism for UNIX design philosophy, as well. And LaTeX code is especially hard to make portable.
There used to be a similar situation with PDFs, it was really hard to display a PDF embedded in application. Finally, Firefox pdf.js came in and solved that issue.
The only embedded and easy-to-implement standard that describes a ‘document’ is HTML, for now (with Javascript for scripting). Only that it’s not aware of page layout. If only there’s an extension standard that could make a HTML page into a document…
- megane-kun ( @megane_kun@lemm.ee ) English3•9 months ago
I was actually thinking of something like markdown or HTML forming the base of that standard. But it’s almost impossible (is it?) to do page layout with either of them.
But yeah! What I was thinking when I mentioned a LaTeX-based standard is to have a base set of “modules” (for a lack of a better term) that everyone should have and that would guarantee interoperability. That it’s possible to create a document with the exact layout one wants with just the base standard functionality. That things won’t be broken when opening up a document in a different editor.
There could be additional modules to facilitate things, but nothing like the 90’s proprietary IE tags. The way I’m imagining this is that the additional modules would work on the base modules, making things slightly easier but that they ultimately depend on the base functionality.
IDK, it’s really an idea that probably won’t work upon further investigation, but I just really like the idea of an open standard for documents based on LaTeX (kinda like how HTML has been for web pages), where you could work on it as a text file (with all the tags) if needed.
- MonkderZweite ( @MonkderZweite@feddit.ch ) 2•9 months ago
Finally, Firefox pdf.js came in and solved that issue.
Which uses a bloated and convoluted scripting format specialized on manipulating html.
- DigitalJacobin ( @DigitalJacobin@lemmy.ml ) English1•9 months ago
True, but it offered a much more secure alternative to opening up PDFs locally.
- MonkderZweite ( @MonkderZweite@feddit.ch ) 1•9 months ago
I don’t think so. pdf.js has all few monts a new XSS CVE, which is a web thing only. And if you use anything other than Adobe Reader/Acrobat…
- erogenouswarzone ( @erogenouswarzone@lemmy.ml ) English9•9 months ago
Bro, trying to give padding in Ms word, when you know… YOU KNOOOOW… they can convert to html. It drives me up the wall.
And don’t get me started on excel.
Kill em all, I say.
- Björn Tantau ( @bjoern_tantau@swg-empire.de ) 64•9 months ago
zip or 7z for compressed archives. I hate that for some reason rar has become the defacto standard for piracy. It’s just so bad.
The other day I saw a tar.gz containing a multipart-rar which contained an iso which contained a compressed bin file with an exe to decompress it. Soooo unnecessary.
Edit: And the decompressed game of course has all of its compressed assets in renamed zip files.
- KSP Atlas ( @KSPAtlas@sopuli.xyz ) 2•9 months ago
.tar.xz masterrace
- d_k_bo ( @d_k_bo@feddit.de ) 1•23 days ago
This comment didn’t age well.
- DigitalJacobin ( @DigitalJacobin@lemmy.ml ) English58•9 months ago
This is the kind of thing i think about all the time so i have a few.
- Archive files:
.tar.zst
- Produces better compression ratios than the DEFLATE compression algorithm (used by
.zip
andgzip
/.gz
) and does so faster. - By separating the jobs of archiving (
.tar
), compressing (.zst
), and (if you so choose) encrypting (.gpg
),.tar.zst
follows the Unix philosophy of “Make each program do one thing well.”. .tar.xz
is also very good and seems more popular (probably since it was released 6 years earlier in 2009), but, when tuned to it’s maximum compression level,.tar.zst
can achieve a compression ratio pretty close to LZMA (used by.tar.xz
and.7z
) and do it faster[1].zstd and xz trade blows in their compression ratio. Recompressing all packages to zstd with our options yields a total ~0.8% increase in package size on all of our packages combined, but the decompression time for all packages saw a ~1300% speedup.
- Produces better compression ratios than the DEFLATE compression algorithm (used by
- Image files:
JPEG XL
/.jxl
- “Why JPEG XL”
- Free and open format.
- Can handle lossy images, lossless images, images with transparency, images with layers, and animated images, giving it the potential of being a universal image format.
- Much better quality and compression efficiency than current lossy and lossless image formats (
.jpeg
,.png
,.gif
). - Produces much smaller files for lossless images than AVIF[2]
- Supports much larger resolutions than AVIF’s 9-megapixel limit (important for lossless images).
- Supports up to 24-bit color depth, much more than AVIF’s 12-bit color depth limit (which, to be fair, is probably good enough).
- Videos (Codec):
AV1
- Free and open format.
- Much more efficient than x264 (used by
.mp4
) and VP9[3].
- Documents:
OpenDocument / ODF / .odt
- @raubarno@lemmy.ml says it best here.
.odt
is simply a better standard than.docx
.
it’s already a NATO standard for documents Because the Microsoft Word ones (.doc, .docx) are unusable outside the Microsoft Office ecosystem. I feel outraged every time I need to edit .docx file because it breaks the layout easily. And some older .doc files cannot even work with Microsoft Word.
- @raubarno@lemmy.ml says it best here.
- By separating the jobs of archiving (
.tar
), compressing (.zst
), and (if you so choose) encrypting (.gpg
),.tar.zst
follows the Unix philosophy of “Make each program do one thing well.”.
wait so does it do all of those things?
- DigitalJacobin ( @DigitalJacobin@lemmy.ml ) English15•9 months ago
So there’s a tool called tar that creates an archive (a
.tar
file. Then theres a tool called zstd that can be used to compress files, including.tar
files, which then becomes a.tar.zst
file. And then you can encrypt your.tar.zst
file using a tool called gpg, which would leave you with an encrypted, compressed.tar.zst.gpg
archive.Now, most people aren’t doing everything in the terminal, so the process for most people would be pretty much the same as creating a ZIP archive.
- By separating the jobs of archiving (
- Laser ( @Laser@feddit.de ) 7•9 months ago
By separating the jobs of archiving (.tar), compressing (.zst), and (if you so choose) encrypting (.gpg), .tar.zst follows the Unix philosophy of “Make each program do one thing well.”.
The problem here being that GnuPG does nothing really well.
Videos (Codec): AV1
- Much more efficient than x264 (used by .mp4) and VP9[3].
AV1 is also much younger than H264 (AV1 is a specification, x264 is an implementation), and only recently have software-encoders become somewhat viable; a more apt comparison would have been AV1 to HEVC, though the latter is also somewhat old nowadays but still a competitive codec. Unfortunately currently there aren’t many options to use AV1 in a very meaningful way; you can encode your own media with it, but that’s about it; you can stream to YouTube, but YouTube will recode to another codec.
- DigitalJacobin ( @DigitalJacobin@lemmy.ml ) English4•9 months ago
The problem here being that GnuPG does nothing really well.
Could you elaborate? I’ve never had any issues with gpg before and curious what people are having issues with.
Unfortunately currently there aren’t many options to use AV1 in a very meaningful way; you can encode your own media with it, but that’s about it; you can stream to YouTube, but YouTube will recode to another codec.
AV1 has almost full browser support (iirc) and companies like YouTube, Netflix, and Meta have started moving over to AV1 from VP9 (since AV1 is the successor to VP9). But you’re right, it’s still working on adoption, but this is moreso just my dreamworld than it is a prediction for future standardization.
- Laser ( @Laser@feddit.de ) 3•9 months ago
Could you elaborate? I’ve never had any issues with gpg before and curious what people are having issues with.
This article and the blog post linked within it summarize it very well.
- tal ( @tal@lemmy.today ) 0•9 months ago
Encrypting Email
Don’t. Email is insecure . Even with PGP, it’s default-plaintext, which means that even if you do everything right, some totally reasonable person you mail, doing totally reasonable things, will invariably CC the quoted plaintext of your encrypted message to someone else
Okay, provide me with an open standard that is widely-used that provides similar functionality.
It isn’t there. There are parties who would like to move email users into their own little proprietary walled gardens, but not a replacement for email.
The guy is literally saying that encrypting email is unacceptable because it hasn’t been built from the ground up to support encryption.
I mean, the PGP guys added PGP to an existing system because otherwise nobody would use their nifty new system. Hell, it’s hard enough to get people to use PGP as it is. Saying “well, if everyone in the world just adopted a similar-but-new system that is more-amenable to encryption, that would be helpful”, sure, but people aren’t going to do that.
- Laser ( @Laser@feddit.de ) 0•9 months ago
The message to be taken from here is rather “don’t bother”, if you need secure communication use something else, if you’re just using it so that Google can’t read your mail it might be ok but don’t expect this solution to be secure or anything. It’s security theater for the reasons listed, but the threat model for some people is a powerful adversary who can spend millions on software to find something against you in your communication and controls at least a significant portion of the infrastructure your data travels through. Think about whistleblowers in oppressive regimes, it’s absolutely crucial there that no information at all leaks. There’s just no way to safely rely on mail + PGP for secure communication there, and if you’re fine with your secrets leaking at one point or another, you didn’t really need that felt security in the first place. But then again, you’re just doing what the blog calls LARPing in the first place.
- MonkderZweite ( @MonkderZweite@feddit.ch ) 4•9 months ago
.odt is simply a better standard than .docx.
No surprise, since OOXML is barely even a standard.
is av1 lossy
- DigitalJacobin ( @DigitalJacobin@lemmy.ml ) 13•9 months ago
AV1 can do lossy video as well as lossless video.
wait im confusrd whats the differenc ebetween .tar.zst and .tar.xz
- DigitalJacobin ( @DigitalJacobin@lemmy.ml ) 8•9 months ago
Different ways of compressing the initial
.tar
archive.
- ronweasleysl ( @ronweasleysl@lemmy.ml ) English2•9 months ago
Damn didn’t realize that JXL was such a big deal. That whole JPEG recompression actually seems pretty damn cool as well. There was some noise about GNOME starting to make use of JXL in their ecosystem too…
- Archive files:
- Supermariofan67 ( @Supermariofan67@programming.dev ) 34•9 months ago
Ogg Opus for all lossy audio compression (mp3 needs to die)
7z or tar.zst for general purpose compression (zip and rar need to die)
why does ml3 need todie
- Supermariofan67 ( @Supermariofan67@programming.dev ) 10•9 months ago
It’s a 30 year old format, and large amounts of research and innovation in lossy audio compression have occurred since then. Opus can achieve better quality in like 40% the bitrate. Also, the format is, much like zip, a mess of partially broken implementations in the early days (although now everyone uses LAME so not as big of a deal). Its container/stream format is very messy too. Also no native tag format so it needs ID3 tags which don’t enforce any standardized text encoding.
- jtfletchbot ( @jtfletchbot@lemmy.ko4abp.com ) 6•9 months ago
Not the original poster, but there are newer audio codecs that are more efficient at storing data than mp3, I believe. And there’s also lossless standards, compared to mp3’s lossy compression.
why does zip and rar need to die
- Supermariofan67 ( @Supermariofan67@programming.dev ) 8•9 months ago
Zip has terrible compression ratio compared to modern formats, it’s also a mess of different partially incompatible implementations by different software, and also doesn’t enforce utf8 or any standard for that matter for filenames, leading to garbled names when extracting old files. Its encryption is vulnerable to a known-plaintext attack and its key-derivation function is very easy to brute force.
Rar is proprietary. That alone is reason enough not to use it. It’s also very slow.
- jtfletchbot ( @jtfletchbot@lemmy.ko4abp.com ) 8•9 months ago
Again, I’m not the original poster. But zip isn’t as dense as 7zip, and I honestly haven’t seen rar are used much.
Also, if I remember correctly, the audio codecs and compression types. The other poster listed are open source. But I could be mistaken. I know at least 7zip is and I believe opus or something like that is too
- MonkderZweite ( @MonkderZweite@feddit.ch ) 3•9 months ago
Most mods on Nexus are in rar or zip. Also most game cracks; or as iso, which is even worse.
- jtfletchbot ( @jtfletchbot@lemmy.ko4abp.com ) 1•9 months ago
That would explain why I don’t see them often. I haven’t been very active in gaming as of late, let alone modding. And I generally don’t pirate games. I’m cool with people that do, I just don’t personally. (Virus fears, being out of the loop long enough that I don’t know any good sites, etc)
- PlexSheep ( @PlexSheep@feddit.de ) 3•9 months ago
How about tar.gz? How does gzip compare to zstd?
- Supermariofan67 ( @Supermariofan67@programming.dev ) 3•9 months ago
Both slower and worse at compression at all its levels.
- drwankingstein ( @drwankingstein@lemmy.dbzer0.com ) English2•9 months ago
its worth noting that aac is actually pretty good in a lot of cases too
- Supermariofan67 ( @Supermariofan67@programming.dev ) 4•9 months ago
However, it is very patent encumbered and therefore wouldn’t make for a good standard.
- drwankingstein ( @drwankingstein@lemmy.dbzer0.com ) English1•9 months ago
aac lc and he-aac are both free now hev2 and xhe aren’t, but those have more limited use
- Aatube ( @Aatube@kbin.social ) 2•9 months ago
What’s wrong with mp3
- Knusper ( @Knusper@feddit.de ) 16•9 months ago
Big file size for rather bad audio quality.
- MonkderZweite ( @MonkderZweite@feddit.ch ) 1•9 months ago
How about xz compared to zstd?
- Supermariofan67 ( @Supermariofan67@programming.dev ) 2•9 months ago
At both algorithms’ highest levels, xz seems to be on average a few percent better at compression ratio, but zstd is a bit faster at compression and much much faster at decompression. So if your goal is to compress as much as possible without regard to speed at all,
xz -9
is better, but if you want compression that is almost as good but faster,zstd --long -19
is the way to goAt the lower compression presets, zstd is both faster and compresses better
- Elise ( @xilliah@beehaw.org ) 33•9 months ago
I wish there was a more standardized open format for documents. And more people and software should use markdown/.md because you just don’t need anything fancier for most types of documents.
- raubarno ( @raubarno@lemmy.ml ) 12•9 months ago
Yes, but only if everyone adhere to CommonMark version of Markdown.
- holland ( @holland@lemmy.ml ) 3•9 months ago
Nah, Pandoc Markdown is the true path.
why, what even is markdown
- perviouslyiner ( @perviouslyiner@lemm.ee ) English5•9 months ago
Standardized open format for documents might have been the only ISO meeting where people were protesting in the streets - https://en.wikipedia.org/wiki/Standardization_of_Office_Open_XML
So now ISO officially has two standard formats for the exact same thing!
- Elise ( @xilliah@beehaw.org ) 2•9 months ago
Impressive! Thanks for sharing. I didn’t know there was a standard. So if someone sends me a docx I can ask them for an iso format now :)
- MonkderZweite ( @MonkderZweite@feddit.ch ) 1•9 months ago
Well, actually ODF came first, to which MS paniked (governments liking standards), so they then remade their office formats to OOXML. Except most of it proprietary extensions, 600 page confusing documentation, bribery during standardization (ECMA, they failed ISO), etc.
- monobot ( @monobot@lemmy.ml ) 5•9 months ago
I agree, we need support for it in libreoffice and than other document editors.
We can not expect people to use codes, but editor that saves to it would be grat.
- IsoKiero ( @IsoKiero@sopuli.xyz ) English33•9 months ago
I don’t know what to pick, but something else than PDF for the task of transferring documents between multiple systems. And yes, I know, PDF has it’s strengths and there’s a reason why it’s so widely used, but it doesn’t mean I have to like it.
Additionally all proprietary formats, specially ones who have gained enough users so that they’re treated like a standard or requirement if you want to work with X.
- kkard2 ( @kkard2@lemmy.ml ) English13•9 months ago
oh it’s x, not x… i hate our timeline
- StarkillerX42 ( @StarkillerX42@lemmy.ml ) 8•9 months ago
I would be fine with PDFs exactly the same except Adobe doesn’t exist and neither does Acrobat.
- darklamer ( @darklamer@lemmy.dbzer0.com ) 4•9 months ago
When PDF was introduced it made these things so much better than they were before that I’ll probably remain grateful for PDF forever and always forgive it all its flaws.
- StarkillerX42 ( @StarkillerX42@lemmy.ml ) 3•9 months ago
I would be fine with PDFs exactly the same except Adobe doesn’t exist and neither does Acrobat.
- sunbeam60 ( @sunbeam60@lemmy.one ) 33•9 months ago
SQLite for all “I’m going to write my own binary format because I is haxor” jobs.
There are some specific cases where SQLite isn’t appropriate (streaming). But broadly it fits in 99% of cases.
- Spore ( @Spore@lemmy.ml ) 1•9 months ago
Also parquet if the data aren’t mutated much.
give me a category please
- sunbeam60 ( @sunbeam60@lemmy.one ) 2•9 months ago
I’ll take “what’s that file format for $300 please”
- MonkderZweite ( @MonkderZweite@feddit.ch ) 1•9 months ago
Yeah, what was it? If office formats used sqlite instead of zip?
- d_k_bo ( @d_k_bo@feddit.de ) 31•9 months ago
JPEG-XL for rasterized images.
- hikaru755 ( @hikaru755@feddit.de ) 4•9 months ago
Never heard of that, thanks for bringing it to my attention!
- 𝕽𝖚𝖆𝖎𝖉𝖍𝖗𝖎𝖌𝖍 ( @sxan@midwest.social ) 25•9 months ago
Resume information. There have been several attempts, but none have become an accepted standard.
When I was a consultant, this was the one standard I longed for the most. A data file where I could put all of my information, and then filter and format it for each application. But ultimately, I wanted to be able to submit the information in a standardised format - without having to re-enter it endlessly into crappy web forms.
I think things have gotten better today, but at the cost of a reliance on a monopoly (LinkedIn). And I’m not still in that sort of job market. But I think that desire was so strong it’ll last me until I’m in my grave.
- glibg10b ( @glibg10b@lemmy.ml ) 13•9 months ago
JPEG XL for images because it compresses better than JPEG, PNG and WEBP most of the time.
XZ because it theoretically offers the highest compression ratio in most circumstances, and long decompression time isn’t really an issue when the alternative is downloading a larger file over a slow connection.
Config files stored as serialized data structures instead of in plain text. This speeds up read times and removes the possibility of syntax or type errors. Also, fuck JSON.
I wish there were a good format for typesetting. Docx is closed and inflexible. LaTeX is unreadable, inefficient to type and hard to learn due to the inconsistencies that arise from its reliance on third-party packages and its lack of guidelines for their design.
- davefischer ( @davefischer@beehaw.org ) English6•9 months ago
TeX / LaTex documentation is infuriating. It’s either “use your university’s package to make a document that looks like this:” -or- program in alien assembly language.
I like postscript for graphic design, but not so much for typesetting. For a flyer or poster, PS is great.
- nothendev ( @nothendev@sopuli.xyz ) 4•9 months ago
Typst for typesetting. Definitely underrated.
- AlexWIWA ( @AlexWIWA@lemmy.ml ) English11•9 months ago
Markdown for all rich text that doesn’t need super fancy shit like latex
- morrowind ( @morrowind@lemmy.ml ) 1•9 months ago
I’d argue asciidoc is better, but less well known
- Lemmy ( @Lemmy@iusearchlinux.fyi ) 1•9 months ago
asciidoc lost me because it’s not a markdown superset. Why invent yet another way of marking headlines?
Also GitLab/Hub markdown is the standard and I don’t think we need another.
- morrowind ( @morrowind@lemmy.ml ) 1•9 months ago
That’s a weird way of thinking. I could make the reverse argument.
Markdown lost me because it’s not a subset to asciidoc, why invent yet another way of marking headlines?
Also asciidoc is the standard and I don’t think we need another.
This whole thread is discussing ideal standards.
- drwankingstein ( @drwankingstein@lemmy.dbzer0.com ) English11•9 months ago
matroska for media, we already have MKA for audio and MKV for video. An image container would be good too.
mp4 is more prone to data loss and slower to parse, while also being less flexible, despite this it seems to be a sort of pseudo standard.
(MP4, M4A, HEIF formats like heic, avif)
wait why not av4 or jpegxl
- drwankingstein ( @drwankingstein@lemmy.dbzer0.com ) English5•9 months ago
those are media formats, not containers.
- Turun ( @Turun@feddit.de ) 1•9 months ago
A mp4 file contains media in, for example, h264 and AAC codec, which is the combined for playback. It is not a codec itself.
im compiling summarised list in body, what do i put this under and what file extensions
- nothacking ( @nothacking@discuss.tchncs.de ) 11•9 months ago
UTF-8 for plain text, trying to figure out the encoding, especially with older files/equipment/software is super annoying.
i’d like there to be a way to standardise midi info in plugins for music