Use of cookie jar
Eric — Dec 22, 2017 10:17AM CET
I use term-extraction of which feed reader is part. I see that there is a library wich contains a script to use a cookie jar. But I do not find any documentation on how to use this. The comments in the script are not enough for me.
In the Netherlands we have very strict cookie rule for websites so a cookie jar is essential.
Do you have some info in using this feature?
2 Community Answers
Keyvan Minoukadeh - Dec 22, 2017 at 11:51AM CET
Cookie Jar is used by our HTTP library (Humble HTTP Agent) to store cookies sent in the HTTP response. These are only stored temporarily for use in (if any) subsequent requests resulting from the original request. Often sites will set a cookie and issue a HTTP redirect and the stored cookie must then be sent to the new URL for the request to succeed. That's the purpose of CookieJar.php.
I suspect the cookie situation in the Netherlands you refer to is a slightly different one: visitors having to accept that a site will be using cookies before the actual content is sent. From the examples I've seen, this is often an interstitial webpage with a 'continue' link or a 'accept cookies' link leading to the actual content. In Full-Text RSS, we deal with these pages using site configuration files. So, for example, if we know that a website, let's say example.nl is presenting these interstitial cookie warnings pages, we create a site configuration file called example.nl.txt with something like the following:
#bypass cookie check
single_page_link: //a[contains(@href, '/cookiewall/accept')]
Here we tell Full-Text RSS to look for a link on the page containing '/cookiewall/accept' in the URL and to follow it.
You'll find real examples used in Full-Text RSS here:
Hope that's some help.
Robin - Mar 19, 2018 at 08:57PM CET
I found a few sites that require forms to be submitted for the cookies to be accepted. Is there a way to configure this in the site config?
<form method="post" action="https://tweakers.net/my.tnet/cookies/" id="cookieAcceptForm">
<button class="fancyButton" name="decision" value="accept" type="submit" title="Ja, ik accepteer cookies">Ga verder</button> <a class="textButton" href="https://tweakers.net/cookies/" style="margin-left:10px">Meer informatie</a></span>
<input type="hidden" name="returnTo" value="YjdmNjNmYTQxOWU1ODk5ZWZhN2NhNDMwY2ExMjhjNDQyMTk0ZTczNA|https://tweakers.net/my.tnet/cookies/">
<input type="hidden" name="fragment" value="">
<input type="hidden" name="tweakers_token" value="AgBecTeKWrALTXr2IOh4THDFMTplnBFfn8NPMAbt">