Proxy configuration in Feed Creator

Storm Proxies guide

We have a guide on the blog (opens new window) on configuring Feed Creator and Full-Text RSS to use the Storm Proxies rotating proxy service.

If you host Feed Creator yourself, you can configure it to use proxy servers. It can be configured to work in a number of modes:

• Never use proxy
• Always use proxy
• Only use proxy when &proxy request parameter is used

A proxy server should only be needed in situations where a direct connection doesn’t work (see our blog post (opens new window) for more details), so we recommend option 3 above. To enable that mode, you'll want to edit the config file.

We recommend you copy the config file into the custom/ folder and edit it there, so that future updates to the software don’t overwrite your settings. Feed Creator will override config settings with those found in the custom/config.php file when it runs.

There are 3 configuration options in the config file that you’ll want to edit. Let’s go through them one by one.

Step 1: Enter proxy server IP and port

Find the IP address and port number for the proxy service you're using.

$options->proxy_servers = ['my-proxy' => ['host' => 'x.x.x.x:xxxxx']];

Make sure to edit the above and replace 'x.x.x.x:xxxxx' with the IP address and port (number after the colon) of your proxy service.

TIP

If you’re using a different proxy service that requires username and password for authentication, you can add that with the ‘auth’ key, e.g.: ['host'=>'x.x.x.x:xxxxx', auth'=>'user:pass']

Step 2: Disable proxy use by default

We don’t want the proxy server to be used for every request, so let’s disable it by default.

$options->proxy = false;

Step 3: Allow proxy use per request

But we do want to be able to indicate that the proxy server should be used for certain requests, so we want to enable proxy override.

$options->allow_proxy_override = true;

Using the proxy service

With the above changes, you won’t notice any difference when using Feed Creator, and won't see an option for choosing a proxy server in the interface.

To indicate that the proxy server you entered should be used, you will need to change the feed URL generated by Feed Creator and add &proxy=my-proxy as a request parameters.

Example

.../feed-creator/extract.php?url=example.org...&proxy=my-proxy

More than one proxy service?

To use different proxy services for different feeds, you can enter more in the configuration file and give each one a unique name. Then pass the name of the proxy service that should be used in the proxy request parameter. So some feeds might have &proxy=proxy1 and others &proxy=proxy2.

Here’s how the configuration might look:

$options->proxy_servers = [
  'proxy1' => ['host' => 'x.x.x.x:xxxxx'], 
  'proxy2' => ['host' => 'x.x.x.x:xxxxx', 'auth' => 'user:pass']
];

Testing the proxy service

The best way to test if the proxy service is working is by fetching a page which shows you the IP address making the request. We’ll use myexternalip.com (opens new window) here. If you load that now, it should show you the IP address associated with your current connection.

When using Feed Creator, the IP address shown from such a page will be the IP address of the server running Feed Creator. When the same request goes through a proxy service, the IP address shown will be one connected to the proxy service, not your server.

Testing Feed Creator

1. Enter the URL https://myexternalip.com/ into the URL field in Feed Creator and hit ‘Preview’
2. You should see the IP address of the server Feed Creator is hosted on in the results:
3. “My External IP address – [your server IP]”
4. Click the RSS Feed button to load the RSS feed in your browser
5. Edit the URL in your browser’s address bar based on the instructions in the previous section, so the final URL will look something like: .../feed-creator/extract.php?url=myexternalip.com&proxy=my-proxy
6. Now you should be shown a different IP address:
7. “My External IP address – [IP associated with proxy service]”