IP Address Reflector

2019-01-08: ~~It appears that this has gone down! I will investigate as soon as I am able.~~ Aaaand we’re back!

2018-03-07 Update: I have recently moved house. Yay! One downside is that I now have a really slow upload speed (0.8mbps on a good day). Please don’t hammer this poor little ADSL line into oblivion! ~~I have also migrated this website over to the same Raspberry Pi 2 that runs this tool, as the old server was incredibly power hungry~~ The site itself now runs on a VPS!

2019-04 Update: I’ve had to unplug this due to the ongoin renovations. I’ll get it back online or migrate it to an online VPS at some point.

Now on a VPS!

Visit

In the browser:

ip.fyr.io

Or from the command line:

curl ip.fyr.io

2019 update

I’ve moved this tool to a VPS now, due to the house move mentioned in the updates above. The downside to this is that I’m unable to customise the installation to quite the level I was hoping to reach. Therefore, some of the below no longer applies. I’ll leave it, as it may be useful if you’re technically curious and want to know some details about HTTP, configuring NGINX, etc.

TL;DR:

ip.fyr.io is a simple, script friendly IP reflector – it’ll tell you your public IP address and nothing more! Feel free to use it – ~~it runs on a Raspberry Pi 2, so be kind or you might crash it. Saying that,~~ there are no limits – no request limits, no extra data to parse away (outside of the essential HTTP headers), and the access logs don’t exist (the error logs do though, for diagnostic problem solving reasons.)

If you want to use it in a script, the only things to be aware of are:

Yes, it is standards compliant.
Both HTTP 1.1 and HTTP 1.0 are supported. The main difference between the two is that HTTP 1.1 GET requests respond using Transfer-Encoding: chunked whereas HTTP 1.0 GET requests do not, as per RFC 2616.
HEAD requests are supported – HEAD responses contain your IP address in the X-Requester-IP: field.

If you want more information, I’ve written up my journey in creating this simple tool below.

Let me know if there are any improvements I can make (or mistakes I have already made below) by commenting on this article.

Enjoy!

The Problem

On many occasions I’ve needed a way to determine my externally facing IP address – often the environment or the network I or a script is running from changes and it’s too much hassle to manually check/update the IP. Therefore, I opted to solve my problem by asking a server to read the address back to me.

There are plenty of sites out there that do this already, but most of the commonly used ones have… issues. Typically, websites like whatismyipaddress.com fill their page with unneeded and unwanted crud – HTML elements filled with images and advertising, javascript libraries and CSS files being pulled down from anywhere and everywhere, you’ll also often find their layout changing every few months, meaning you have to revisit your scripts in order to fix what is now broken code – all very script-unfriendly. Not only that but most of these websites have rate limiters or other awkward limitations on how many times you can pull your IP. This can be worked around but it’s just more complicated than it really should be.

There are exceptions to this, beacons of light in a sea of darkness: donavans ip reflector on strewth.org is one, which returns you IP address and FQDN json encoded. There are more alternatives out there!

Donavans and other reflectors aside, I’ve opted to create my own. Another can’t hurt, right? Redundancy, yay!

Requirements

So, here’s what I want it to do:

DO: Tell me my external, publicly facing IPv4 Address

Pretty simple. I want to add IPv6 support too. In fact, it should work already, I just don’t have an IPv6 address with my ISP, but I can get one, I just need to ask! Here’s what I don’t want:

DON’T: Artificially restrict or limit anyone
DON’T: Keep access logs
DON’T: Be difficult to parse (by humans or code)

Should be fun!

Construction

So I began building. Here’s the process I went through to get from initial conception to the current state of the utility. We’ll start with the tools used and move on to the build itself.

Tools used

My requirements dictate that a script or user should be able to get their public IP address from within any network. The best (outside of documentation or checking a config somewhere, this is the least effort and most trustworthy) way to know your public IP is to use the network in question to access something external you trust, and have that external thing tell you the IP it received the request from.

In almost all networks, port 80/HTTP (and often 443/HTTPS, but more on that later) is open on the firewall, optionally with a web filter in place between the user or script and the Internet itself. I could run my reflector off of a non-default port, but then you’re requesting firewall reconfigurations from the network/security team, as well as having to remember some awkward port number. For such a simple task, we may as well just host the reflector as a website on port 80. This has the added advantage of working for users who need to know their public IP quickly without wanting to check documentation, check configurations or deal with the messy pages on existing websites. All they need is a web browser and knowledge of the URL.

It’ll be an incredibly simple website, so doesn’t need any kind of powerful hardware behind it. I have a spare Raspberry Pi 2 around somewhere, so I can host it on that. It’s a good opportunity for me to have a play around with Nginx, too.

As for the how of getting the users IP reflected, we may as well settle with PHP, simply because it’s easy and I’m lazy. We could use pretty much any other language here, though.

So, in summary, this is what we’ve got to work with:

A single Raspberry Pi 2 running Rasbian 7
Nginx
PHP

Reflector Script

So, first off, let’s imagine I have a script sat somewhere behind the misty depths of a NAT router. This script needs to know its publicly facing IP address for whatever reason and as a lazy scripter all I want to be told is the IP address with reliability. Nothing more. Looking at the server side, this is easily done with PHP: <php echo $_SERVER['REMOTE_ADDR'];
That will return the IP address (v4 or 6) that the request came from. If we were operating behind a proxy or load balancer we could have also used $_SERVER['HTTP_X_FORWARDED_FOR']
to (potentially) determine the original IP address, but for this usage scenario it isn’t required – we’re interested in the IP address that the world sees when you access the Internet. I may revisit this in the future, though.

There is a potential security issue with using $_SERVER['REMOTE_ADDR']; on some websites in specific scenarios as the information can be spoofed, but as we’re simply echoing back the contents of the variable to the requester (spoofed or not) and not using it in any meaningful way, we don’t need to worry about it. If the source IP address is somehow spoofed at the TCP level (which won’t really happen) and a TCP session is successfully created, the only impact would be my server sending unwanted data to the IP used in the spoofing attempt. Considering that this will be running on a Raspberry Pi, I suspect my machine will crash long before the attack has any significant impact, or indeed is even noticed.

I am using single quotes ( ‘ ) opposed to double quotes ( ” ) within the $_SERVER['REMOTE_ADDR'] variable to decrease execution time. Using double quotes forces PHP to check for special characters or strings within the text itself – any variables will be pulled and replaced with their contents and any escape sequences will be interpreted. Admittedly, the time saved by using the single quotes here is negligible, but every little helps.

I have omitted the closing PHP tag simply because it is best practice. There is the added bonus that it makes the script slightly quicker to execute, what with PHP not having to parse the two extra characters. Like the single/double quotes mentioned above, this is also a negligible efficiency enhancement.

So, we have our script ready! Time to prepare the Raspberry Pi 2.

A Slice of Pi

I ordered this Pi 2 back when they first became available (remember the Xenon flash bug?) but haven’t managed to do anything with it yet, except for a couple of small PHP experiments which have faded from memory.

I have a MicroSD card with Raspbian 7 already on it – probably from a previous endeavor – so will just use that for now. In the future I will likely gift the Pi 2 a fresh installation of Raspbian Jessie Lite, which is essentially Raspbian but without the desktop. A headless installation would no doubt provide greater efficiency and allow for higher traffic before struggling so it’s a no-brainer, but as this is just a proof-of-concept for now it doesn’t matter too much.

Luckily, one of the username/password combinations I tried from memory worked and I was able to log into the Pi 2 over SSH. After resetting this password, I could then quickly install Nginx and PHP by following this guide. Normally in these kinds of articles you expect an in-depth explanation of problems faced and the demons I had to fight away in order to get it all working right, but everything worked perfectly the first time! The only thing left to do is sort out the configuration file.

The Engine (x)

The Nginx configuration is similar to Apache in that it uses the sites-enabled/ symlinks to load the configuration of the websites it hosts. For this project I’ve opted to contain my configuration in the /etc/nginx/sites-available/default file. My gut feeling is that it will be slightly faster to process one slightly larger file than several smaller ones, even though it’s running off an SD card with relatively zero seek time compared to a spinning disk. I have no idea if this is the case though, hopefully I’ll revisit it in the future.

I have set up a new server block at the bottom of the default configuration file, which is pasted below:

# ip.fyr.io
server {
  server_name ip.fyr.io;

  root /usr/share/nginx/ip.fyr.io/www;
  index index.php;

  location ~ \.php$ {
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    fastcgi_index index.php;
    include fastcgi_params;
  }
}

The default configuration listens on port 80 by default so all we’re doing here is essentially saying If a request comes in with the “Host” header field set to “ip.fyr.io”, present them with the specified directory root. If no file is requested, load “index.php” by default. Pass files ending with “.php” to php5-fpm. I did think about leaving the server_name variable out so that any requests that reach this device on port 80 (regardless if they come via the hostname with the “host” header set or just the IP address, etc) are routed to the IP reflector, but I want to use the Pi 2 later on to host more tools I have planned – using the hostname to separate these just makes it easier.

As an aside, this plus the decision to use Nginx has a downside – it forces you to make a valid HTTP request instead of just sending a null probe to port 80… though the fact that you’re sending and receiving valid HTTP traffic over port 80 shouldn’t trip any IDS/IPS sensors, so doing it this way, whilst slightly increasing the complexity for you (you’ll have to roll your own (or load an external) basic HTTP library if your language doesn’t have one built in) I feel the pros outweigh the cons.

HTTPS

It’s worth mentioning here that I haven’t opened or configured port 443 (HTTPS) for this domain. The reason being that it doesn’t really need it, and will only slow things down. There’s a brief note in the Todo section below about it. That’s not to say it shouldn’t have it, but I’ll be adding it at a later date with LetsEncrypt.

Testing and refining

I have moved the index.php file created earlier across to the Pi 2 and added a subdomain to the fyr.io zonefile, opened up the port on the firewall and waited a few hours for the DNS changes to propagate.

The first test is to use curl from the pi itself:

dev@raspberrypi ~ $ curl ip.fyr.io
261.74.13.81dev@raspberrypi ~ $

The result of the query appears on the second line – because the server doesn’t return a newline after the IP address, my username immediately follows after the last octet. The important thing is that the IP is correct! I gave it a quick test from my phone over 4G which works as expected, the result matching what I’m seeing on a couple of other IP address sites.

But we’ve still got some efficiencies to make before we can call it a day!

I open up WireShark to capture a request from my desktop to ip.fyr.io and the servers response – first, here’s the request sent by Firefox to the server containing the request header:

GET / HTTP/1.1
Host: ip.fyr.io
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive</code></pre>
<p>...and here's the response from the server.</p>
<pre><code lang="wireshark-capture">HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Wed, 03 Aug 2016 19:28:06 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.45-0+deb7u4
Content-Encoding: gzip

20
...........0.97.24..82....!.....
0

The request and response can be broken into two parts – the header, and the body (in the case of our request, we only have the header, not the body.) These sections are separated by a blank line. We don’t really care about the content of the request as that is something outside of our control – Two things are immediately apparent with the response, however:

There are a few headers that we don’t need to see in there
Where’s the IP address in the body?

The headers should be easy enough to deal with, likely by editing the Nginx config file, so we’ll investigate that in a minute, but what’s going on here with the body?

Efficiency: The Response Body

The website itself shows a plain IP address! Visiting the website in the browser shows the correct information, so something must be going inside the browser itself.

Take a look at the headers in the response and you’ll notice that Content-Encoding: has gzip set – this means the HTTP response has been compressed using the gzip algorithm.

To save bandwidth, the body of the HTTP response is generally compressed, but this actually works against us due to the tiny amount of data we’re sending back. So, I’ve disabled gzip compression on the server by adding the line gzip off – let’s see what Wireshark captures:

HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Thu, 04 Aug 2016 19:50:59 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.45-0+deb7u4

c
82.68.178.20
0

You can see that there are fewer characters to transmit in the body now. Generally for any decent volume of text you’ll want to leave compression on but for the tiny amount of data here it ends up being more expensive (computation- and bandwidth-wise) than it’s worth.

In the first capture, there were three lines of data, the first and third being numbers (“20” and “0”, respectively.) I had put this down to the gzip compression at first, but we’re seeing it again here (albeit with different values.) After some investigating, it turns out this is due to Transfer-Encoding: chunked.

If that header is present, the body is sent in chunks – blocks of data at a time. Before each chunk is a hexadecimal value that specifies the length of the following chunk. In our captured data we can see the ‘c’ character, which, converted to decimal, equals ’12’. This tells the browser that the following chunk will be 12 bytes long – each character in our response is one byte, so that means there should be 12 characters in the IP address shown in the example, which is indeed the case.

When you count the number of characters in the capture above, you can ignore any newlines as we only transmit a single line of data – if you wanted to be accurate (and that’s why we’re here!) you would need to account for the \r\n (carriage return and new line feed) immediately following the chunk size, as well as the \r\f that occurs just after the end of the chunk. Do not count these characters, but do count any that occur within the chunk itself. What follows the actual chunk is the chunk size for the next chunk, and so on until we hit a chunk size of 0, which indicates the end of the data.

We will only ever send a single chunk, so if we want to remove these extra characters can we switch over to using the Content-Length: XYZ response header, which specifies the size of the body? For large response bodies, transmitting chunks of data is better, but for us, will using the older Content-Length header work out to be more efficient?

For now, I have decided to leave the chunked transfer encoding enabled – it is in the HTTP 1.1 spec, so if you’re making HTTP 1.1 requests, you (or the library/tool you’re using) should support it. I may investigate switching over to Content-Length in the future.

If you don’t mind handling chunked responses, just perform a GET request using HTTP 1.1. If you don’t want to bother with it, use HTTP 1.0, as chunked encoding is forbidden in RFC2616

Efficiency: The Response Headers

We’ve already looked at some of the response headers whilst sorting out the body of the reply – here’s what it currently looks like when an average web browser visits ip.fyr.io:

HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Tue, 09 Aug 2016 18:06:56 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.45-0+deb7u4

Let’s go through some of this and see what can be done to reduce the size of the response and make it slightly more efficient.

`HTTP/1.1 200 OK`

This first line of the response is called the Status-Line which tells the client the protocol version (“HTTP/1.1“) followed by the status code (“200“) and its associated human-readable phrase (“OK“), which means “everything processed fine, here’s the result of your request”.

This line is required by the specification, so we’ll have to keep it.

`Server: nginx/1.2.1`

The second line displays some information about the server that processed our request. In our case it is telling us that the hosting software is nginx and its current version is 1.2.1.

The spec has a note that encourages server implementors to “make this field a configurable option” so we can remove it, as long as Nginx allows us to!

There’s good news and bad news – it looks like Nginx doesn’t have the option to hide the Server: response header without editing and compiling the source code or installing extra modules, but we can modify the response header slightly to remove the version information, thus reducing the size of the response, by adding server_tokens off to the Nginx config file. In doing this, the line in question becomes Server: nginx

`Date: Tue, 09 Aug 2016 18:48:17 GMT`

To be investigated in a future post.

`Content-Type: text/html; charset=utf-8`

To be investigated in a future post.

`Transfer-Encoding: chunked`

To be investigated in a future post.

`Connection: keep-alive`

To be investigated in a future post.

`X-Powered-By: PHP/5.4.45-0+deb7u4`

This is a thing PHP chucks in there. We can easily turn it off by changing expose_php On to expose_php Off in the php.ini file. In our case, we want to edit the file in /etc/php5/fpm/php.ini. I had to reboot the pi for this to take effect. I tried restarting the relevant services but it wasn’t having it. Very odd, I’m sure I missed something simple there.

Log files

There are a couple of types of log files we’re concerned about with this setup, the Nginx access_log and the error_log.

They’re self explanatory – access_log logs access, error_log logs errors. I want to know about errors and be able to diagnose them, so I want to keep this enabled, however the access log can be removed entirely!

We can do this by opening up the /etc/nginx/nginx.conf file and looking for the line access_log /var/log/nginx/access.log;. Comment this out by placing a # at the start, then add the following line underneath:access_log off;. Job done! Whilst we’re here, we may as well delete all the existing access_log file located in /var/log/nginx. Navigate to the directory, then run sudo rm access_log*.

Wrapping up for now

Turning something small into something smaller and more efficient managed to be a much bigger task than expected. As a result, I’m leaving it there for now. There are some things to solve, some kinks to iron out as the saying goes, so I will revisit this in the future.

TODO:

There’s a lot to investigate here, but for now it’s working fine. You can find a list of notes below of things I wanted to investigate and write about but didn’t have time. At some point I’ll revisit this and make things better!

~~Investigate $_SERVER[‘HTTP_X_FORWARDED_FOR’] stuff~~ – might be useful, but will ignore for now
~~Write a bit about setting the Pi up~~ – done
Re-add references
Write about nginx:
- General thoughts
- ~~Disabling gzip compression for this server_name block~~
- why no HTTPS? (Because it’ll make things a teeny bit slower, more resource intensive, and has no privacy advantages, but it could stop injection… hrm. Part 2?)
- the config itself
- ~~disable/rotate logging~~ – Access log is disabled, error log is still active.
- favicon/404 errors? This’ll use up some resources
~~Currently ip.fyr.io uses “Windows-1252” character encoding? Should I change this to UTF-8? What do other browsers report? also see http://htmlpurifier.org/docs/enduser-utf8.html~~ – I have added “charset utf-8;” to the nginx default file for ip.fyr.io and it has changed the charset, but added some extra data to the response header. It is still 12 bytes.
Investigate removing http headers for the cleanest possible output – Partly done, this could be a part 2?

2017-10-07 Add Comment

In Technology

IP Address Reflector

Visit

2019 update

TL;DR:

The Problem

Requirements

Construction

Tools used

Reflector Script

A Slice of Pi

The Engine (x)

HTTPS

Testing and refining

Efficiency: The Response Body

Efficiency: The Response Headers

HTTP/1.1 200 OK

Server: nginx/1.2.1

Date: Tue, 09 Aug 2016 18:48:17 GMT

Content-Type: text/html; charset=utf-8

Transfer-Encoding: chunked

Connection: keep-alive

X-Powered-By: PHP/5.4.45-0+deb7u4

Log files

Wrapping up for now

TODO:

Leave a Reply Cancel reply

`HTTP/1.1 200 OK`

`Server: nginx/1.2.1`

`Date: Tue, 09 Aug 2016 18:48:17 GMT`

`Content-Type: text/html; charset=utf-8`

`Transfer-Encoding: chunked`

`Connection: keep-alive`

`X-Powered-By: PHP/5.4.45-0+deb7u4`