I didn’t explain what I meant very well. To scrape a website you don’t need to understand robots.txt, implementing robots.txt is something you do to be a good netizen. But to get like info from Lemmy, implementing ActivityPub is a requirement.
Now I’ll admit, it’s not a great system and I do wish we had something better, but I also don’t think “this isn’t a good way to communicate preferences” is a good reason to ignore them.
The comparison doesn’t work because both Lemmy and Mbin are implementing the same standard, while robots.txt is mostly an honour system.
Information not being private isn’t the same thing as information being public.
idk, the label is also an honor system, if it can be just ignored like robots.txt.
I didn’t explain what I meant very well. To scrape a website you don’t need to understand robots.txt, implementing robots.txt is something you do to be a good netizen. But to get like info from Lemmy, implementing ActivityPub is a requirement.
Now I’ll admit, it’s not a great system and I do wish we had something better, but I also don’t think “this isn’t a good way to communicate preferences” is a good reason to ignore them.