Salta al contenuto principale


!Friendica Support

So I realise that there are instances out there that allow stuff I don't want to see (ever).

I'm good at ignoring stuff, so whatever, BUT...

I just saw a post in the global feed that had such an image, but being served *from* my server.

Now I'm assuming it's cached it for some reason, but why is that cache exposed to users who aren't local (ie. Logged in)?

How can I find out how an image ended up on my server?

If it was because of one of my users, they are getting booted.

If not, how can I find out what instance the image is from in general (eg. If I can't find the post but have the image URL)?

I'd like to identify the source and remove any other images from it, it's gross, and probably illegal.
in reply to Ian Molton

Images are always served from the local machine because they are cached. When it was created by a remote user then you have got the possibility to block that user.
in reply to Michael Vogel

@Michael Vogel ok. But given the image URL, how can I find out who caused it to be uploaded / cached?

How can a remote user cause friendica to cache an image?

If the latter is possible, that's a real problem, a bit like an open relay, but for images, and I don't want people filling my servers cache with... That. Eww.
Questa voce è stata modificata (1 anno fa)
in reply to Ian Molton

More to the point, how could the poster have got the image cached, and how did they figure out the URL that points to it on my server?
in reply to Ian Molton

I guess that we misunderstand each other. From my point of view someone did a post that was received by your machine. Upon arrival the external url is transformed into a local url. So when you see some external post with a local url then this is totally okay. Furthermore: This data isn't stored on your system, it is only relayed. So when you delete that post then all traces are gone.
in reply to Michael Vogel

@Michael Vogel ok, so my first question is (unless one of my users is doing something they shouldn't), why did my server receive the post? I assume that it doesn't just accept posts if none of my users requested the post in question?

Secondly, why transform the URL? What's the benefit?

Surely it's better for my users machines to fetch the images wherever they are, rather than to centralise that traffic at my server?

Lastly, is it possible for me to take an image URL and undo the transformation, so I can find out where it points to?

Lastly, just an observation, but if the images really aren't cached, it seems likely to cause alarm (as it has in this instance) when you see kiddie porn with your instances domain in it. Surely the displayed URL should reflect the true origin of the image?
in reply to Ian Molton

The caching is done to avoid the possibility that remote pages could fetch data about the visitors.

The origin url of attached media can be seen in the database. But this doesn't matter. It will be some server where the poster had access to. You can block that user server wide. And you can check the server where the post came from to tell the administrator about it.
in reply to Michael Vogel

@Ian Molton Remember that this excludes embed videos like from # and # ... You still may block them with e.g. ||youtube.com^$third-party so they won't load on your instance as this might reveal your referral URL + UA string + IP address to these external sites.
in reply to Ian Molton

Hm. I think something is being lost here; Im probably not explaining well.

Here's a (NSFW, no idea what the japanese text reads) example URL:

https://friendica.mnementh.co.uk/photo/media/40044

I can access this *without being logged in*, so presumably anyone on the internet can see this image.

This URL was being shared in a post that looked like this:

If I hover over the image in the post itself, the URL shown is:

https://img.pawoo.net/media_attachments/files/004/365/347/original/09a0b05d478957fd.jpg

From my perspective, it looks like someone is using my instance to store, or hide a link to, NSFW/potentially illegal images, by tricking it into *publicly* caching the images, and then sharing the URL in their posts.

I find this behaviour disturbing in the extreme - I don't want my instance to contain (or appear to be the source of) such content, and I *especially* don't want it to get blocked because someone else is sharing links to its cached content!

in reply to Ian Molton

This is a misunderstanding. When you go directly to that page then you see that the picture is stored on pawoo.net. The link .../photo/media/... on your own post is just the link to your local content redirector. There is no misuse of your server.
in reply to Michael Vogel

@Michael Vogel

I wouldn't mind if that was only available to local users - but non-logged-in users can access this, and they have (afaict) no way to know this isnt coming direct from my server.

And someone has clearly (see screenshots) made a post somewhere that links *directly* to my server.

It doesn't matter if the content is on my server or not - I don't want it *publicly* even reflecting these URLs. Why would it need to? My users should have access to such redirections (on legitimate, legal posts), but the rest of the world, I would rather keep out!

especially seeing as they can apparently guess the URLs that the reflector / cache / whatever uses.

IOW - I dont really care if its legal or illegal to have links to such content on my server - I *dont* want them. Ever. In the UK, being accused of having this sort of thing is enough to get you lynched. I *do not* want it.

This is what people see if they follow the link:

They will neither know, nor care wether I actually have a copy of the image, and I would really rather not have to move country because of a badly configured cache/reflector/redirect thats open to abuse, as it clearly is.

in reply to Ian Molton

@Ian Molton@Michael Vogel The cache at /photos/ is there to prevent leaking referral URLs the user-agent string and your IP address to these external sites, like pawoo.net is. If you don't want any # pictures on your instance, simply block all those domains, you can check through mine if you want a quick start. Please note that not all blocked domains are NSFW-related.
in reply to Roland Häder

@Roland Häder@Michael Vogel

Right - so the cache is there for the benefit (exclusively) of my users, right?

If so, why is it *ever* accessible to a non-logged in user?

I've blocked both pawoo.net and baraag.net as they are loli and shota (two new words for me) tolerant instances, and I am not.

But that's just going to be playing whack-a-mole - others will pop up, and whilst I've now set up a few saved searches for stuff I dont want to have anything to do with...

...it doesn't prevent the problem mentioned earlier, in that *literally* ANYONE on the internet can see such content, apparently originating from my server (as best as *they* can tell.

At *best*, its a waste of bandwidth.

At worst, whilst I might not go to prison over it, I'd potentially end up in court, and have my reputation ruined.

Is there really *any* reason why the cache has to be accessible to the outside world, ever?
in reply to Ian Molton

@Michael Vogel@Roland Häder

Oh, and lastly, I'd still like to know if its possible to find out WHY my instance cached that URL - *I* certainly never interacted with that profile, and I would *hope* none of my handful of users have, so why did it get cached in the first place?

(or, if one of my users is a pervert, I'd really like to get them *off* my instance, like now. AFAIK, I don't have "common carrier" status in the UK, like the big corporates.)
in reply to Ian Molton

Do you have a robots.txt that forbids crawling? If yes, then there shouldn't be any reason for non users to have a look at your public feed. These people more easily could have a look at the public feed of those systems. Also in the admin settings you can forbid access to the community pages.
in reply to Michael Vogel

@Michael Vogel@Roland Häder

I don't think I've been understood.

However the image URLs ended up on my system is immaterial.

the fact is that my instance was serving up porn if people followed those URLs.

Someone was also making posts elsewhere, with links *pointing to those URLs on my instance*.

I presume they were doing this to hide the origin of the illegal material they had managed to trick my instance into caching.

IOW, my friendica instance became an "open relay" for kiddy porn.

Wether its "earier" for someone to distribute their kiddy porn some other way or not is not the point here.

The fact is that they were doing it, and my server was participating in it.

This is BAD no matter what.

Now, either one of my users is a pervert, in which case, I'd like to know how I can find that out (is there a quick db query I can do? presumably there is, but I'm not familiar enough with the db to know) Can you tell me how I can find that out?

OR

My users arent perverts, and someone else has found a way to get my server to cache things no-one asked for.

This is also BAD.

I don't understand why this isn't a cause for concern. My friendica node is not special, and pretty much a default configuration. If this happened to me, it could happen to anyone.

I'm starting to be very worried about security of the Friendica server.
in reply to Ian Molton

@Ian Molton Yes, "guest" (not logged-in) can see your pictures. Yes, other instances will come and go. Just search for NSFW-related keywords and keep blocking them, this is how I do it here. People can watch (porn/nudity) whatever they want, but not on my instance.
in reply to Roland Häder

@Roland Häder

I don't want to be / sound rude, but I've said this several times now...

These Are Not (really REALLY not) My Pictures.

REPEAT:

THESE ARE NOT MY PICTURES.

I have NO IDEA why Friendica cached those URLs, and I'm *extremely* alarmed that I cannot seem to find that out, or no-one seems to know / care.

I dont want to go to PRISON. Especially not for *that* sort of thing, and *REALLY* especially not in the UK.

If Friendica REALLY cannot be set up NOT to act as a **kiddie-porn reflector**, then I am going to have to abandon it, which would be a great shame, as I otherwise like it quite a lot.
in reply to Ian Molton

@Ian Molton I understand your concern. Just block and ignore them, they try to bully you maybe. Here I do this and so far nobody has bothered me with that I host pornography on my instance.
in reply to Roland Häder

@Roland Häder

My concern is simple.

Living in a country where you can be lynched (murdered by a mob) for hosting that kind of content, the ABSOLUTE LAST thing I want is for my friendica server to be seen to be serving up kiddie porn to anyone who asks it nicely.

Now, if it's one of my users, I want to know who.

How can I find out?

If it's not, then I want to know *why* it's caching and serving up URLs to kiddie porn.

Because it could literally get me killed.

If you don't live in the UK, you won't understand.

But I'm serious.

Besides, being an open relay is just anti-social behaviour in the internet.

Honestly this thread has really made me think twice about continuing to host a friendica instance.

If the Devs don't understand the security implications of an open relay, then there is no hope for the project.

I hope this is still all a miscommunication. Because if not, that's scary AF.
in reply to Ian Molton

@Roland Häder there are roving gangs of "pedo hunters" in the UK.
Questa voce è stata modificata (1 anno fa)
in reply to Ian Molton

What is the legal position if you end up unintentionally hosting criminal content in a P2P system? I know Section 230 provisions or equivalent apply to the likes of defamation on a web platform, but what if it is distributing content that it's a *crime* to possess. I'm concerned the law might not be reasonable on this point as well.
in reply to Alastair Cooper

@Alastair Cooper

I don't know.

What I do know is that I *REALLY* dont want to find out. Getting taken to court isnt my concern.

Its what the UK *public* do to people where there is even a *rumour* that something has happened.

I dont know if the @Friendica Support forum is the right place to raise this concern, but I do want people to realise that for people living / hosting a sever in the UK, this stuff is *SCARY* dangerous even to talk about.

And very easily taken out of context.

And the public *do not care* if you get acquitted - child sex related stuff sticks, and will stick to you FOR LIFE here.

Friendica devs - this is a serious, serious issue.
in reply to Ian Molton

I don't know the UK law. In the EU you mustn't store such content on your systems (which we don't do). And AFAIK you just have to remove posts like these when you are notified about them.

But like I said: You can disallow the public timeline for visitors and for example only display posts from local users.
in reply to Michael Vogel

@Michael Vogel@Alastair Cooper

Again. I am not concerned about my (or my other users) timelines.

Im concerned that my friendica node is sharing kiddie porn that I do not want it to.
in reply to Ian Molton

Your node isn't sharing that content. It received it.
in reply to Michael Vogel

@Michael Vogel@Alastair Cooper

I'm sorry, but it very much was sharing it.

To all and everyone on the internet.

Can you not see my screenshots?
in reply to Ian Molton

@Michael Vogel@Alastair Cooper it's not sharing it *now* because I've blocked and purged the offending instances.

But it very much was, and it was not requiring a login (I logged out and used a private tab).
in reply to Ian Molton

That post was visible on the community page that was visible also for non logged in visitors. But your server hadn't shared it to other servers.
in reply to Michael Vogel

@Michael Vogel@Alastair Cooper

Ah, I think I see the miscommunication then.

By shared with the world, I mean "made accessible to the world” (not logged in users), not shared with other friendica servers.

I don't want my server to make it's cache available to the world at large. Only to my users.

Then this can never happen.
in reply to Ian Molton

Let me try to answer you.

Someone on your server interacted with an account on pawoo.net, whether by following this account, or fetching a public post (by entering its URL in the search bar) or subscribing to a hashtag if you have configured your node to accept content from post relays (although it appears unlikely).

pawoo.net then sends this post including a link to an image hosted on pawoo.net to your node. To protect your user's privacy, your node will download the image locally and rewrite the URL in the post to point at its location on your node.

So you are right, your node will be publishing this image from your domain until you take a moderation action against the post itself, the remote account (node-wide account blocking includes an option to delete all the content) or the remote server (node-wide server blocking includes an option to delete all the remote accounts and their content.

This local copy is meant to protect your user's privacy, but unfortunately it can't be limited to users at it stands. Since the same exact process is used to publish your user's own pictures in their public posts, making it logged-in user only would prevent their public posts to show their images to non-logged users. Think about RSS users or simply their public profile page.

Now, as Michael mentioned, there are ways of limiting the risks for you as a website publisher. Moderation is the main point (and the URL of the picture you mentioned now returns "Not Found") but as you said it is a game of whack-a-mole. It is made easier by domain blocking but there will be an amount of time during which questionable content will be published by your node.

Making the community pages (either from your local users or the global network your node knows about) private to your users only is another way of limiting exposure, as only your users will be able to be exposed to this kind of content through images your node is publishing. But then they would be able to share the image URL publicly for anyone else to see.

Currently images know whether they should be served to anyone or just to your users based on the permissions of the posts their are embedded in but I guess we could improve this system to take the site setting to make the community page private into account. This would further reduce the risk for you as even if one of your users shared the URL of such a picture with a non-logged in user they wouldn't be able to access it.
in reply to Hypolite Petovan

@Hypolite Petovan@Ian Molton Hypolite, can you help me find the "accept content from post relays" setting? thank you very much for your time

Friendica Support reshared this.

in reply to Luca Nucifora

It's here in the admin section: https://friends.polli.social/admin/site#admin-settings-relay-collapse

It also includes instructions on how to subscribe to relays.

Friendica Support reshared this.

in reply to Hypolite Petovan

Parola filtrata: nsfw

Friendica Support reshared this.

in reply to Roland Häder

Parola filtrata: nsfw

Friendica Support reshared this.

Unknown parent

Hypolite Petovan
There seems to be a technical misunderstanding. Cookies are arbitrary data set by a website on a user's browser for ulterior retrieval. Authentication cookies are typically used to persist a successful login on a given website for a given period of time. Server to server communication do not use cookies and cannot set cookies in any user's browser.

We do have a mechanism to authenticate a Friendica user on a remote node, but it involves several requests back and forth between the two nodes so it isn't really suited to authenticate image display.

Friendica Support reshared this.

Unknown parent

Ian Molton
@Matthew Exon@Hypolite Petovan

Surely an interim solution would be to serve local users from the cache, but serve the original URL to everyone else?

Friendica Support reshared this.

in reply to Ian Molton

Also, i'd like to change my vote for code change priorities.

Id like to see this fixed before anything else :-)
Questa voce è stata modificata (1 anno fa)

Luca Nucifora doesn't like this.

Friendica Support reshared this.

in reply to Ian Molton

It is possible, but this would require to alter a post content depending on the authentication status and this would yield a significant performance hit for all post display operations.

A less costly solution would be to redirect anonymous users to the original URL but the image URL with your domain would still be visible in the source of the post page, I don't know how tech-savvy are your roving pedo hunter gangs.

Friendica Support reshared this.

in reply to Hypolite Petovan

@Hypolite Petovan@Matthew Exon I can't imagine "if local user, serve cache URL, else serve original URL" should have a huge performance hit.

Likewise, when serving from the cache, only serve if local user.

Or is it impossible for the server to tell the difference between a cached image and a legitimate image belonging to a local user?

Friendica Support reshared this.

in reply to Ian Molton

The devil is in the details. Computing a post display from a raw post body is a costly operation; we need to convert Markdown/BBCode tags to HTML, resolve mentions, rewrite image URLs... so we only do it once and store the result that we use for everybody, logged or not. Discriminating between logged in and anonymous users would require us to perform the same operation twice, store the result in parallel, and pick the correct version when building a page. This is possible, but not insignificant for overall performance.
Or is it impossible for the server to tell the difference between a cached image and a legitimate image belonging to a local user?
It is possible, it would be part of the solution I suggested earlier, it just hasn't been implemented yet.

Friendica Support reshared this.

in reply to Ian Molton

@Hypolite Petovan@Matthew Exon ah, you did say that above, my bad.

So can we have a "remote" flag or something attached to an image, which is set for images that are cached for local users, and is unset, should a local user actually make a post with that image?

That wouldn't prevent anyone seeing it if a user shared the image, but at least from a legal point of view, it's more clearly their fault, and easy for me to fix (by booting them)

Friendica Support reshared this.

in reply to Ian Molton

Yes, the flag would work in conjunction with the setting for hiding the global community page from anonymous users. If the page is hidden, the image URL wouldn't work for anonymous users, and if the page is shown, the image URL would work for everybody.

Friendica Support reshared this.

Unknown parent

Ian Molton
@Matthew Exon@Hypolite Petovan

the pedo hunters are, frankly, blithering idiots, and they aren't above planting evidence (this has ruined many police investigations).


There are definitely people who will sell them tools to do this.

So this is a very real threat imo.

Friendica Support reshared this.

Unknown parent

Hypolite Petovan
What does "bulletproof server" means?

Friendica Support reshared this.

Unknown parent

Hypolite Petovan
Got it. I'll explore the feasibility of both solutions (redirect and settings-based) this week.

Friendica Support reshared this.

Unknown parent

Ian Molton
@Matthew Exon@Hypolite Petovan I wouldn't have worried if not for actually seeing a post in the wild which directly referenced those URLs.

The people who share that kind of stuff will also share URLs out-of-band, so this is already a real world attack, rather than theoretical.

Really pleased to see this is high on @Hypolite Petovan 's priorities, that's all anyone can ask for :)
Questa voce è stata modificata (1 anno fa)

Friendica Support reshared this.

Unknown parent

Hypolite Petovan
Thank you for the elaboration, I want to take roving gangs of pedo hunters seriously.

Friendica Support reshared this.

in reply to Ian Molton

What tickles me in this issue is that image caching is supposed to be an asset, not a liability. It would be simpler for us to keep the original image URLs but we added this processing to protect Friendica users' privacy. This side-effect of changing the ownership of questionable material is typical of a feature which was only considered from one angle. and I'd like to correct the record.

Friendica Support reshared this.

Unknown parent

Hypolite Petovan
What do you mean by "delegating to the proxy"?

Friendica Support reshared this.

Unknown parent

Hypolite Petovan
How would this look for embedded images?

Friendica Support reshared this.

Unknown parent

Hypolite Petovan
Then what’s the situation where we get to rewrite the image link but not the displayed link?

Friendica Support reshared this.