Express parameter callbacks

Handy little feature I didn’t know about in Express:
Using app.param([name], callback) you can bind callbacks directly to route parameters, allowing you to move common preprocessing/validation out of each function that uses the parameter, and into a single function (without having to call it explicitly each time.)
You can pass in an array of names, using next() to jump to the next parameter, and the callback is only called once regardless of how many times the parameter appears in route handlers.
The callbacks are local to the router they are defined on, so you can handle things (or not) differently based on the context.
Neat!

Redis Cluster vs Redis Replication

While researching Redis Cluster I found a large number of tutorials on the subject that confused Replication and Cluster, with people setting up ‘replication’ using cluster but no slaves, or building a ‘cluster’ only consisting of master-slave databases with no cluster config.

So to clear things up:

Replication

Replication involves a master server which serves reads and writes, and duplicates all data to one or more slave servers (which serves reads but not writes). Slaves can be used to replace a master in case of failure, spread read request load, or to perform backups of the database to reduce load on the master.

Cluster

Clusters are used when you have more data than RAM in a single machine: the data is automatically split (based on the key) across multiple databases, increasing the amount of data you can store. Clients requesting a key from any cluster node will be redirected to the node holding the key, and are expected to learn the locations of keys to reduce the number of redirects.

Replicaton + Cluster

Redis Cluster supports replication by adding slaves to existing nodes, if a master becomes unreachable then its slave will be promoted to master.

Sentinel

Last but not least, Redis Sentinel can be used to manage replicated servers (not clustered, see below.) Clients connect to a Sentinel and request a master or slave to communicate with, the sentinels handle health checks of the masters/slaves, and will automatically promote a slave if a master is unreachable. You need to have at least 3 sentinels running so that they can agree on reachability of nodes, and to ensure the sentinels aren’t a single point of failure.

Cluster handles its own promotion and does not need Sentinel in front of it.

Installing Luasec part 2: Failed loading manifest

Problem

Tried to install LuaSec on a new machine recently and got the following error:

luarocks install luasec
Warning: Failed searching manifest: Failed loading manifest: Failed fetching manifest for http://luarocks.org/repositories/rocks – Error fetching file: Failed downloading http://luarocks.org/repositories/rocks/manifest – URL redirected to unsupported protocol – install luasec to get HTTPS support.

So all I need to do to install LuaSec is install LuaSec first,  brilliant!

Solution

One solution is buried here https://github.com/luarocks/luarocks-site/issues/6

and is to specify the server directly:

luarocks install –only-server=http://rocks.moonscript.org luasec

Stuck installing debuginfo in Ubuntu

Problem:

To run systemtap you need debuginfo, but it fails when installing linux image with :

apt-get source linux-image-4.4.0-53-generic-dbgsym
Reading package lists… Done
Picking ‘linux’ as source package instead of ‘linux-image-4.4.0-53-generic-dbgsym’

And then fails to find ‘linux’

Solution:

Solution is to uncomment ‘deb-src’ in /etc/apt/sources.list, run apt-update again, and then

sudo apt-get build-dep –no-install-recommends linux-image-$(uname -r)

error: ‘struct module’ has no member named ‘symtab’

Problem:

Running system tap gives the error: ‘error: ‘struct module’ has no member named ‘symtab’’

Solution:

This is caused by a bug with system tap not containing symtab for ubuntu 16+ in version 2.9, and can be solved by upgrading to systemtap 3.0+ by compiling from source.

Openresty/NGINX accessing Docker environment variables

Problem:

Docker supports passing in environment variables to your containers as a handy way to easily switch environments when using multiple docker-compose files. For example you may have a base docker-compose.yml with a docker-compose.dev.yml and docker-compose.prod.yml file that specify environment variables for database hostnames/credentials.
The issue is simply accessing these environment variables directly isn’t possible in openresty (e.g. using os.getenv()).

Solution:

The issue is caused by NGINX blocking all environment variables by default. To make them accessible again you need to re-define them in your nginx.conf, but with no values, e.g:

env APP_ENV;

You should then be able to access them as normal.

Zabbix Aggregate Checks and “Unsupported Item Key”

Problem:

Trying to create an aggregate check in Zabbix to measure total bandwidth in a group of servers using ‘grpsum’, get status ‘Not supported’ and error message ‘Unsupported item key.’

Solution:

It’s not anywhere obvious in the docs  or after googling the error, but the item type needs to be changed from ‘Zabbix agent’ to ‘Zabbix aggregate’

Steam big picture controller shortcut windows 10

Problem:

Installed Windows 10, installed steam, xbox button on controller doesn’t open up Big Picture any more

Solution:

Need to disable DVR setting in the xbox application that comes with win10.

Can either login into the xbox app and disable the DVR option under settings, or if you can’t login for some reason, you can press the windows key and g in any program, and select “this is a game”, then turn off the dvr setting there.

 

Syntax error sending emails with Lua and Gmail

Got a fairly unhelpful error while trying to send an email using LuaSec and LuaSocket to connect to googles SMTP servers:

555 5.5.2 Syntax error. uc3sm9880237wib.2

Turns out google doesn’t like the recipient format

‘FirstName LastName <name@example.com>’

and instead prefers just

‘<name@example.com>’

Minecraft Monitoring with Zabbix Part 2

This is just a quick update on the previous post about monitoring zabbix, and assumes you already have zabbix set up and monitoring the basic properties of your server. (if you don’t, see here.)

We’re going to add a simple php script on our minecraft server that queries the number of players online, then return this value to the zabbix server by adding it ot the agent as a UserParameter.

Adding the PHP:

No point in reinventing the wheel, so we’re going to grab  xPaw’s MinecraftQuery.class.php from here and save it on the minecraft server as “MinecraftQuery.class.php”

Then we need to add our own file which uses this one, queries our server ip and port, and returns the number:

<?php
require_once(‘MineCraftQuery.class.php’);

$Query = new MinecraftQuery( );

try
{
//$arg[1] is the argument passed in when calling the php, in this case the port
$Query->Connect( ‘127.0.0.1’, $argv[1] );

$result = $Query->GetInfo( );

echo $result[“Players”];
}
catch( MinecraftQueryException $e )
{
//return 0 if something goes wrong
echo 0;
}

unset($Query);
unset($result);
?>

Now we add a custom user parameter to the bottom of /etc/zabbix/zabbix_agentd.conf :

UserParameter=minecraft.players[*],php /home/mc/ServerQuery.php $1

The first part is the key to identify the item in zabbix server: the * in brackets tells the agent to use the values that the server is sending in place of any numbers prepended with $. This means with a single UserParameter on the agent, we can query multiple minecraft servers from the zabbix server, simply by passing a different port in with the trigger. The second part is the command to run our script, change it to match your script’s path.

Finally, on the server we need to reload the zabbix agent so it know about the new key:

/etc/init.d/zabbix-agent restart

Adding the Host Item

In the zabbix web interface, go to configuration -> Hosts, select your minecraft server and go to Item -> Create Item.

For the key, enter the key you used as a UserParamter, but replace the * with the server query port.

Thats all you need to do to add the item, but to make it a bit neater you can set the Units to “players” and create a new application group such as “minecraft players”

Here’s mine:

Capture

Repeat the above for any extra servers you have, and add all of the items to a graph to display them together (see part 1 for more on graphs) You should end up with something like this:

Capture

To link externally to your graphs you need to enable guest readonly access, and use the following address:

http://mydomain.com/zabbix/chart2.php?graphid=<graphID>

(you can find the graph ID in the URL when you are configuring the graph)