How do I set the user-agent of the full text rss crawler?
Thanks.
How do I set the user-agent of the full text rss crawler?
Thanks.
Hi Trung,
You can set the user agent using site config files.
For example, you can add the following line to a site config file to send ‘PHP/5.6’ as the user agent when Full-Text RSS requests content from that site:
http_header(user-agent): PHP/5.6
We have more information about site config files here: http://help.fivefilters.org/customer/en/portal/articles/223153-site-patterns