Basic Miner Operation | Helium Documentation (2024)

Basic Miner Operation​

As you can see below, the Miner is central in routing data across the Helium Network. It is one ofthree pieces:

• Packet Forwarder: this is a utility that interacts with the radio front-end and sends and receivesraw radio packets with the Helium Miner
• Miner: the Helium Blockchain comes into the picture here; the Miner is responsible for routingpackets to the appropriate Router and entering into micro-transactions brokered via libp2p
• Router: a Helium compatible LoRaWAN Network Server, basically; this component is interested inreceiving the packets relating to its devices and handles downlink messages when appropriate

In addition to packet routing, the Miner also participates in PoC and submits PoC packets as witnessevents. When challenge, the Miner sends packets to the packet forwarder for transmission.

The Miner connects to other Miners over libp2p where, amongst other things, it is gossiping andsaving blocks, while maintaining a ledger of the blockchain. Constantly syncing the blockchain anddownloading blocks uses significant bandwidth and CPU.

In this guide, we will explain how to get a Docker image of the Helium Miner running on Ubuntu 20.04LTS, and finally some tips on how to interact with a running Miner.

When preparing the Miner for a Hotspot product, you will want to go a little further and follow thesteps outlined here, where you will learn howto integrate it with security modules and BLE.

If you are interested in contributing to Miner development and code-review, please visitthe Miner's repository on Github.

Setting Up Ubuntu for Docker​

Ubuntu is a widely available Linux distribution. Notably, it has an easy-to-use image available forRaspberry Pi 3 and 4, so we use it as an example system. That being said, any ARM64 or AMD64(X86-64) based OS that can run Docker is suitable.

For Raspberry Pi, we recommend running the latest64-bit RaspiOS image. We currently do nothave Docker image support for 32-bit systems, so please double-check that you're using a 64-bitimage. Once you have followed the instructions and are logged into the system, you are ready for therest of this guide.

For most cloud service providers, launching an instance with Ubuntu 20.04 LTS should be fairlystraightforward. With AWS for example, create an EC2 instance running Ubuntu 20.04. A t2.small willrun the miner well once the initial blockchain sync has completed. Once that's launched and you'reconnected, you are ready for the rest of this guide.

First, update the package manager registry:

sudo apt-get update

Then, install Docker:

sudo apt-get install docker.io

To avoid needing to user docker with sudo privileges, add your user to the docker group,replacing $USER with your username:

sudo usermod -aG docker $USER

Log in and out of your account to apply these changes. You are now ready to use Docker!

Port Forwarding​

Before launching the Miner, you will want to configure ports on your network to forward two ports:

• 44158/TCP: the Miner communicates to other Miners over this port. The networking logic knowshow to get around a lack of forwarding here, but you will get better performance by forwarding theport
• 1680/UDP: the radio connects to the Miner over this port. You will not be able to forwardpackets or participate in Proof of Coverage without this

"Forwarding" on the second port is less relevant if you are running a radio packet forwarder on thesame system at the miner (both on a Raspberry Pi, for example). But it is essential if you arerunning a Miner on the cloud for example.

For AWS, for example, you will want to configure the "Security Group" of your EC2 as so:

Run a Docker Container​

Miner releases are available as amd64 and arm64 images on atquay.io. We do not currently provide 32-bitsupport.

Note: on amd64 systems, AVX support is required. Verify that it exists on your host system:

grep avx /proc/cpuinfo

grep avx2 /proc/cpuinfo

If nothing is returned from these commands, your host system does not have AVX support and yourMiner may not be stable.

Start Container​

Before running the container for the first time, it is advisable to pick a 'system mount point`,i.e. a directory in your host system; some long-term miner data is stored there. This allows you toeasily maintain your miner's blockchain identity (i.e. swarm keys) and blockchain state throughminer image updates.

If you are using a Linux system, you can just create a directory in your user's home directory:

mkdir ~/miner_data

If you are using Ubuntu as user ubuntu, this path would now be /home/ubuntu/miner_data. This willbe used later.

Now you can try the run command to start your container for the first time:

docker run -d --init \
--ulimit nofile=64000:64000 \
--env REGION_OVERRIDE=US915 \
--restart always \
--publish 1680:1680/udp \
--publish 44158:44158/tcp \
--name miner \
--mount type=bind,source=/home/ubuntu/miner_data,target=/var/data \
quay.io/team-helium/miner:miner-xxxNN_YYYY.MM.DD

Replace xxxNN with the architecture used, i.e. amd64 or arm64, and with the release date of theimage.

The -d option runs in detached mode, which makes the command return or not; you may want to omitif you have a daemon manager running the docker for you.

The -env REGION_OVERRIDE=US915 tells your miner that you are connecting to a packet forwarderconfigured for the US915 region. You will want to change this to your region i.e.US915 | EU868 | EU433 | CN470 | CN779 | AU915 | AS923 | KR920 | IN865

Note: REGION_OVERRIDE may be completely omitted once your Miner has asserted location and is fullysynced, but leaving it there (as long as the region is properly configured) is not harmful

The --restart always option asks Docker to keep the image running, starting the image on boot andrestarting the image if it crashes. Depending on how you installed Docker in your system, it'llstart on boot. In the AWS AMI above, we use systemd (systemctl status docker to check).

The --publish 1680:1680/udp binds your system port 1680 to the container's port 1680, where theMiner is hosting a packet forwarder UDP server; this is necessary if you want to do any radiointeractions with your miner.

The --name miner names the container, which makes interacting with the docker easier, but feelfree to name the container whatever you want.

The --mount with the parameters above will mount the container's /var/data/ directory to thesystems directory /home/ec2-user/miner_data.

Configure AWS Instance for Sync​

Amazon EC2 instances have CPU usage credits that will easily be depleted during the initial sync ofthe blockchain. To avoid having your instance throttled, you can temporarily uncap your instance bysetting CPU credit usage to unlimited. Once your miner has reached full block height, a t2.smallinstance is sufficient to keep your miner running.

Interact with the Miner within the Container​

You may want to interrogate the Miner or interact with it it as described in theusing a miner section. Docker's exec command enablesthis e.g.

docker exec miner miner info height

In other words, prepend docker exec miner to any of the commands documented in theusing a miner section). Or create an alias such as:alias miner="docker exec miner miner"

Updating the Docker Image​

the releases here. Depending on whether you are runninga miner for fun, to route packets, or to participate in PoC, keeping it updated may be more or lessurgent. Each release tagged on the Github will be on the quay repository. Simply remove the currentimage:

docker stop miner && docker rm miner

And start the container again as described above, but with the new release tag! Thanks to the--mount option, the blockchain data and the miner keys are preserved through updates.

Using the Miner​

These commands will assume you are running in Docker and they have the same prefix to get youexecuting a command within the docker: docker exec miner . If you want to make it easier, you canalways create an an alias such as: alias miner="docker exec miner miner".

Checking the Logs​

This is always helpful to get some idea of what's going on:

docker exec miner tail -F /var/data/log/console.log

Also, if you are particularly interested in radio traffic, it can be helpful to filter for lora asso:

docker exec miner tail -f /var/data/log/console.log | grep lora

Checking the P2P Network​

This is the first health check you can do to see how your Miner is doing. Is it finding other Heliumminers over libp2p properly?

The Helium blockchain uses an Erlang implementation of libp2p. Because weexpect Hotspot hardware to be installed in a wide variety of networking environmentserlang-libp2p includes a number of additions to the corespecification that provides support for NAT detection, proxying and relaying.

The first order of business once the Miner is running is to see if you're connected to any peers,whether your NAT type has been correctly identified, and that you have some listen addresses:

docker exec miner miner peer book -s

You will see an output roughly like the following:

+--------------------------------------------------------+------------------------------+------------+-----------+---------+------------+
| address | name |listen_addrs|connections| nat |last_updated|
+--------------------------------------------------------+------------------------------+------------+-----------+---------+------------+
|/p2p/11dwT67atkEe1Ru6xhDqPhSXKXmNhWf3ZHxX5S4SXhcdmhw3Y1t|{ok,"genuine-steel-crocodile"}| 2 | 4 |symmetric| 3.148s |
+--------------------------------------------------------+------------------------------+------------+-----------+---------+------------+

+----------------------------------------------------------------------------------------------------------------------------+
| listen_addrs (prioritized) |
+----------------------------------------------------------------------------------------------------------------------------+
|/p2p/11apmNb8phR7WXMx8Pm65ycjVY16rjWw3PvhSeMFkviWAUu9KRD/p2p-circuit/p2p/11dwT67atkEe1Ru6xhDqPhSXKXmNhWf3ZHxX5S4SXhcdmhw3Y1t|
| /ip4/192.168.3.6/tcp/36397 |
+----------------------------------------------------------------------------------------------------------------------------+

+--------------------------+-----------------------------+---------------------------------------------------------+------------------------------+
| local | remote | p2p | name |
+--------------------------+-----------------------------+---------------------------------------------------------+------------------------------+
|/ip4/192.168.3.6/tcp/36397|/ip4/104.248.122.141/tcp/2154|/p2p/112GZJvJ4yUc7wubREyBHZ4BVYkWxQdY849LC2GGmwAnv73i5Ufy|{ok,"atomic-parchment-snail"} |
|/ip4/192.168.3.6/tcp/36397| /ip4/73.15.36.201/tcp/13984 |/p2p/112MtP4Um2UXo8FtDHeme1U5A91M6Jj3TZ3i2XTJ9vNUMawqoPVW| {ok,"fancy-glossy-rat"} |
|/ip4/192.168.3.6/tcp/36397| /ip4/24.5.52.135/tcp/41103 |/p2p/11AUHAqBatgrs2v6j3j75UQ73NyEYZoH41CdJ56P1SzeqqYjZ4o | {ok,"skinny-fuchsia-mink"} |
|/ip4/192.168.3.6/tcp/46059| /ip4/34.222.64.221/tcp/2154 |/p2p/11LBadhdCmwHFnTzCTucn6aSPieDajw4ri3kpgAoikgnEA62Dc6 | {ok,"skinny-lilac-mustang"} |
+--------------------------+-----------------------------+---------------------------------------------------------+------------------------------+

As long as you have an address listed in listen_addrs and some peers in the table at the bottom,you're connected to the p2p network and good to go.

If you are having trouble, try forwarding port 44158 to the miner. On AWS, double check yoursecurity group settings.

Checking Block Height​

As long as a genesis block is loaded, this query will work and return height 1 at least:

docker exec miner miner info height

If you are syncing, you should see something like this:

~$ miner info height
6889 280756

The first number is the election epoch and the second number is the block height of your miner. Ifyou just launched an AMI instance, your Miner is been disconnected, or you simply have a slowconnection, you may be a few blocks behind. To check, you can either check the mobile app, checkthe browser-based block explorer, or run a simple curl commandto check in a Terminal:

~$ curl https://api.helium.io/v1/blocks/height
{"data":{"height":280756}}

Backing Up Your Swarm Keys​

Periodically, we may release updates or you may want to migrate your miner from one machine toanother. The most important thing is that you maintain a copy of your miner's private key, i.e.swarm_key. Fun tidbit: this is also how your three-word name is generated.

Assuming you've mounted the Docker image as detailed above, it is located at:

~/miner_data/miner/swarm_key

Another fun tidbit: for production hotspots sold by Helium, the swarm key is stored inside of asecure element and is thus unable to be migrated (or compromised/accidentally lost unlessphysically damaged).

Snapshots​

Snapshots are summaries of the current state of the blockchain. Since you don't need every singleblock ever to sync with the blockchain, it's really just a summary of the last few blocks and theaccount balance of every account on the blockchain.

You generally want to be careful who you accept snapshots from as they might give you an incorrectstate. However, we also store hashes of valid snapshots in the blockchain.

Chain-based​

You can query a running Miner to see what snapshots it (1) knows about and (2) has a copy of:

$ miner snapshot list
Height 731521
Hash <<129,14,18,225,133,83,34,24,205,69,10,10,84,42,129,207,42,186,18,28,192,
 157,187,76,109,233,4,108,198,197,111,176>>
Have true

Height 730801
Hash <<80,90,115,7,10,195,115,217,87,173,24,11,63,116,201,22,150,6,252,168,204,
 60,65,83,106,68,94,19,7,13,72,165>>
Have true

A Miner knows about a snapshot in two ways: it either syncs a block where the snapshot hash andheight have been saved, or the configuration file of the Miner (sys.config) has the blessedheight and hash written into it (ie: blessed_snapshot_block_height andblessed_snapshot_block_hash).

If you have a fully synced Miner, you can extract these values from snapshot list and write theminto the sys.config of another non-fully synced Miner. Restart the Miner to reload thesys.config and it will be able to "quick sync" to that snapshot. To make things go even faster,try manually connecting your Miner to one of your Miners that has the snapshot.

miner peer connect /ip4/192.168.0.0.1/tcp/44158

The only down-side of this approach is that snapshots happen at some rhythm defined by theblockchain (currently 720 blocks). You can use the manual approach to pass around arbitrarysnapshots by hand.

Manual​

You can also manually move a snapshot file around. To take a snapshot:

miner snapshot take /path/to/snapshot

Naming the snapshot something like snap-XXXXXX, where XXXXXX is the height of the blockchain isa typical convention. Also, when using a Docker container, it's usually a good idea to save thesnapshot to /var/data so that you can easily copy the snapshot out of the Docker and into the hostsystem.

Loading the snapshot is as follows:

miner snapshot load /path/to/snapshot

Sometimes the load command times out. If so, try to tail the logs and see if blocks are beingapplied. It's working just fine! It just needs a moment to finish loading the snapshot.

Connecting a Packet Forwarder to the Miner​

You're almost done. The last step after you have your Miner running is to connect to it.

With your favorite editor, open global_conf.json that lives in the packet forwarder directory:

nano ~/sx1302_hal/lora_pkt_fwd/global_conf.json

You want to change the field "server_address" from "localhost" to the IP address of the server, soin this case:

You'll need to restart the packet forwarder for the configuration change to take effect.

To verify that things are working, you can follow the logs on your miner. eg:

docker exec miner tail -f /var/data/log/console.log | grep lora

At the very least, you should see PULL_DATA messages every few seconds. If so, then you've done it!