Use of cookie jar

Full-Text RSS
1

I use term-extraction of which feed reader is part. I see that there is a library wich contains a script to use a cookie jar. But I do not find any documentation on how to use this. The comments in the script are not enough for me.
In the Netherlands we have very strict cookie rule for websites so a cookie jar is essential.
Do you have some info in using this feature?

2

Hi Eric,

Cookie Jar is used by our HTTP library (Humble HTTP Agent) to store cookies sent in the HTTP response. These are only stored temporarily for use in (if any) subsequent requests resulting from the original request. Often sites will set a cookie and issue a HTTP redirect and the stored cookie must then be sent to the new URL for the request to succeed. That’s the purpose of CookieJar.php.

I suspect the cookie situation in the Netherlands you refer to is a slightly different one: visitors having to accept that a site will be using cookies before the actual content is sent. From the examples I’ve seen, this is often an interstitial webpage with a ‘continue’ link or a ‘accept cookies’ link leading to the actual content. In Full-Text RSS, we deal with these pages using site configuration files. So, for example, if we know that a website, let’s say example.nl is presenting these interstitial cookie warnings pages, we create a site configuration file called example.nl.txt with something like the following:

#bypass cookie check
single_page_link: //a[contains(@href, ‘/cookiewall/accept’)]

Here we tell Full-Text RSS to look for a link on the page containing ‘/cookiewall/accept’ in the URL and to follow it.

You’ll find real examples used in Full-Text RSS here:

Hope that’s some help.

3

I found a few sites that require forms to be submitted for the cookies to be accepted. Is there a way to configure this in the site config?

Example:

Ga verder Meer informatie 
			<input type="hidden" name="tweakers_token" value="AgBecTeKWrALTXr2IOh4THDFMTplnBFfn8NPMAbt">
		</form>

Robin

4

It can be done: tweakers track themselves if a special ID (Secure-TnetID) stored in a cookie has accepted the cookie policy.

I have written a simple python-script that:

  • runs once a day
  • gets a new Secure-TnetID
  • accepts the cookie policy using that ID
  • writes a new custom site-config for FTR that includes the new Secure-TnetID in a cookie

So far this seems to be working.