Myth or Fact? 4,000 transactions per second on the private Catapult blockchain

4,000 transactions per second

I am frequently asked if 4,000 transactions per second will be capable on Catapult. I usually answer these questions by telling them “only on private, not on public”. That being said, I have not personally tried to achieve this result so I cannot say I am sure…yet. In this article I explain my journey of trying this out on the open sourced Catapult on a private chain.

Some Background

To achieve 4,000 transactions per second, you would need block intervals of 15 seconds and 60,000 transactions per block. If the interval is set to 1 minute, you would need about 240,000 transactions per block. FYI, NEM v.1 blocks can hold 120 transactions per block.

The Process

When attempting to achieve 4,000 transactions per second, I’ve kept the settings as close to the default configurations to keep this process repeatable.

I ran a Azure VM machine, using size F32s_v2 with the below environment.

  • ubuntu 18.04 LTS
  • git 2.17.1
  • docker 18.06.1
  • docker-compose 1.22.0
  • nodejs v10.11

In my VM I have ports 22, 3000 and 8000 open while running.

  • 22
  • 3000
  • 8000

Launch Catapult

git clone https://github.com/tech-bureau/catapult-service-bootstrap
cd catapult-service-bootstrap
git checkout 77e6cf38a7845194aa2ce72f4ed4d87e5ab791e3
docker-compose -f docker-compose-with-explorer.yml up -d

Done with Alpaca running.

It seems that processing signatures makes the transactions heavier, which will probably inhibit us from reaching optimal performance. So instead, I created a mass amount of pre-signed transactions.

git clone https://github.com/planethouki/yonsen.git
cd yonsen
npm install

I obtained the nemesis address private key since we need accounts with leftover balances. This was done by opening “addresses.yaml” and obtaining all “private” numbers under “nemesis_addresses:”, then creating file “nemesiskeys.json”.

cat ../build/generated-addresses/addresses.yaml
vi nemesiskeys.json
addresses.yaml
nemesis_addresses:
- private: 1947340C6102E18927B98D49FB4A7947AB0C5AFCDB31F34F7EA85209A2252CF2
  public: <something>
  address: <something>
- private: 1627E64F23841C748DB9B1650ADFEC8868003158D7803A0652FE76FBD2D617B0
 <and more>

Copy and paste the above private keys into the below.

nemesiskeys.json
[
    "1947340C6102E18927B98D49FB4A7947AB0C5AFCDB31F34F7EA85209A2252CF2",
    "1627E64F23841C748DB9B1650ADFEC8868003158D7803A0652FE76FBD2D617B0",
    "2D280ADE3C870B2AD6B962DE191743C763823EC07E897846E5398546E9D69EC7",
    "354AF480EE43126FDC3DD11FD0F6537647F8413B7676536DFFFC671F850DF8A4",
    "<and more>"
]

Now we will start processing the function. We’ve automated transactions to be made from the private key we made earlier, and to randomly choose recipients and signees. This process is going to take about 2 hours so brb reading and gaming.

npm run create32

Now we have 8 text files with 320,000 transactions each. The file created is lined with payloads.

payload_320000_8/payload0001.txt
A50000000363C105F26EC44563137F35A3389E17E15541171118CA45A2253BE379226A0E2142F03DF9F54A14140456B35B1154C6D87BC5C9C771AF66D4883DB7263B970FAAD87BCE49947988710B561FF8F197791E82D15FAE02C1FC661913221B3168C5039054410000000000000000068181361300000090D122B0A1031846C76CBC7D59B298B4CE74307D1562F8CEE60100010029CF5FD941AD25D580841E0000000000
A50000000A201FD031F321F37875266BABAB299B18D3D3B07995C40E1C4F45F838DCD8F776DB75EF24FC6807BB9458FD5B578436DFECE17FEEB7393EE8489B8EB84B9708BDC47DA1770D60BE59458955F6FAFEB1BCCAC724068A08B16E709B49B038AC6303905441000000000000000015818136130000009038E6207AD52AAAA352952E40303B9507A9B548AC96D0176E0100010029CF5FD941AD25D5808D5B0000000000
A5000000C19C58720E65C1FE48586B0697EF251E0C1339596F8631535CE721739330F44A1CBCC66B1B01D0D1E52DBC7A963CB86CEA602A220CCBC9F3336E007AAC4F1906A111E52857B11526FC947E5B35BEB120003CF683D8D58C933C66B9F2479E3ED5039054410000000000000000248181361300000090A3ADA63DEB12930DE99439873D54679EEA3D8D50D293FA3C0100010029CF5FD941AD25D580841E0000000000
A5000000D54EFB265418A258BED1E66D5D2AB00DAC428A402500D117D2CD36CA8920A68F06FC88606D0CEC636D73DB4BD698FCB6632CBE5677D7F10288879F52E85FEA0BEF895B241FE52C943732D66D331810F96859AAF5F01B8ABD11B2AB8B5F4725380390544100000000000000004681813613000000901451D6A14DD5998906CA091C7E5F3D941233717EEE3304E70100010029CF5FD941AD25D5C0CF6A0000000000
A500000006347DABEE0F6037957A1E80232BC2DFF32ACA313C1592D16D222634EFED5E651424FF6FE49B1395BF7BA3BF72402CB35108DEBFBEFB23E26D45814B57A35707A594C49AF8E95AE87450D99957BF7052A00CDE4E2345475415C081CCC53F401103905441000000000000000067818136130000009091064DF93DAE8C16706BEDA010AD0E4475B513E3E1CBF2850100010029CF5FD941AD25D580841E0000000000
A500000074662C063BE55D7D01A9A412791E532B70CDCFF93451DBFE5B011F170193FD5A134030BF4148E35B5FC54E310CA4A36CCF314E9838046A9C476450C7C7D7E101BDC47DA1770D60BE59458955F6FAFEB1BCCAC724068A08B16E709B49B038AC63039054410000000000000000908181361300000090D122B0A1031846C76CBC7D59B298B4CE74307D1562F8CEE60100010029CF5FD941AD25D500127A0000000000
A50000000BF8C02E4DFD5A90DF65A7D5B18C701819C2F234FED93CBD6D6D8B01D78ADADED9B0CC316AFCAAE31FBDBDD4CE83646749FB693CF7B23F8CC96A04289AC4500A3548D5BD43DD840726778609A6E5CCAF7359A80E8EF4117F07010CE7E54BFB800390544100000000000000009F818136130000009016CDA788B3EDF7C3498EF5CCDB24885BBB7977FD661B40860100010029CF5FD941AD25D500127A0000000000
A5000000B32236F28B618D562367A2F52362C1311A93D8291AFFD3937840FB1BCC50F7C4C2E2D09CE9216BCBEBAD8890686B653AB2D5F10C7F02343C3C012B405BFFF30B8364AD86104064C268D166E2522A8D88C36FD3BB6FEA9862C679F2706178FBEB039054410000000000000000C281813613000000901451D6A14DD5998906CA091C7E5F3D941233717EEE3304E70100010029CF5FD941AD25D5C0CF6A0000000000
A500000062E538ED968016ABF8500DAE6A727842FB56CF582D5BAF9357D3E23EA677432C81141D11A12C3BC1A8A586966AA46699D4F7B9BBAE3E41D2677400FA374B6D0408F034041AB4C0CB6692985BE06DDB887289F52D2E0A95AFF9C6D2A3BD42055E039054410000000000000000828181361300000090C8453CA8D5FAB0A7363F448311FD0FA266C646D59C9D312B0100010029CF5FD941AD25D580841E0000000000
A50000002A028616EE26B5EC09BC15A8A2742A8CD5E4A1D8494BDB4E4E0D35870CEB14F2DCB1B72456A36C35BA0A20BFB76519462FA7799FAA0100570A1CC4A82DE6830CDCB11E2550BDBF17C7D89E86B0E2C80A5B63DBA187CC1780D905951DF3135915039054410000000000000000E381813613000000909B9DC59709A46FABF6EBE28A80794D443DDAF91FE9DB1D300100010029CF5FD941AD25D5C0CF6A0000000000
...

Sending the transactions

I’ll try sending. 1 first, then sending 500 after. (send 1 and send 500 respectively)

npm run send1
npm run send500

We can keep track of this level of transaction through Block Explorer. (Http-access to VM port 8000 through your browser)

Now let’s try sending 320,000 x 8 transactions. This will probably take about 15 minutes. At this quantity, block explorer will most likely crash so I’ll keep track through the following method.

npm run send32

On Block Explorer, “CONTENT_LENGTH_MISMATCH”, this error stops displaying transactions. So we will use “nem2-sdk” to fetch data and excel to count transactions.

npm run view
Toggle
{"height":1,"timestamp":"2016-04-01T00:00:00.000Z","harvester":"SAXNID-TNOM6A-2JZOWT-SDW4PB-CY5BU2-VWZL75-BFQG","txes":25,"fees":0}
{"height":2,"timestamp":"2018-12-09T05:14:51.094Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":0,"fees":0}
{"height":3,"timestamp":"2018-12-09T05:15:13.097Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":1,"fees":0}
{"height":4,"timestamp":"2018-12-09T05:15:36.100Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":0,"fees":0}
{"height":5,"timestamp":"2018-12-09T05:16:04.103Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":500,"fees":0}
{"height":6,"timestamp":"2018-12-09T05:16:16.106Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":0,"fees":0}
{"height":7,"timestamp":"2018-12-09T05:16:38.108Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":13146,"fees":0}
{"height":8,"timestamp":"2018-12-09T05:17:06.123Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":113285,"fees":0}
{"height":9,"timestamp":"2018-12-09T05:17:30.251Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":98982,"fees":0}
{"height":10,"timestamp":"2018-12-09T05:17:56.340Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":105007,"fees":0}
{"height":11,"timestamp":"2018-12-09T05:18:19.433Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":28303,"fees":0}
{"height":12,"timestamp":"2018-12-09T05:18:41.470Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":153513,"fees":0}
{"height":13,"timestamp":"2018-12-09T05:18:54.594Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":25990,"fees":0}
{"height":14,"timestamp":"2018-12-09T05:19:19.624Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":127507,"fees":0}
{"height":15,"timestamp":"2018-12-09T05:19:41.733Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":43008,"fees":0}
{"height":16,"timestamp":"2018-12-09T05:20:05.772Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":140779,"fees":0}
{"height":17,"timestamp":"2018-12-09T05:20:35.922Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":125231,"fees":0}
{"height":18,"timestamp":"2018-12-09T05:20:47.040Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":28053,"fees":0}
{"height":19,"timestamp":"2018-12-09T05:21:11.064Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":30522,"fees":0}
{"height":20,"timestamp":"2018-12-09T05:21:35.092Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":179782,"fees":0}
{"height":21,"timestamp":"2018-12-09T05:21:56.232Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":18894,"fees":0}
{"height":22,"timestamp":"2018-12-09T05:22:14.254Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":138686,"fees":0}
{"height":23,"timestamp":"2018-12-09T05:22:37.431Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":30691,"fees":0}
{"height":24,"timestamp":"2018-12-09T05:22:51.469Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":118690,"fees":0}
{"height":25,"timestamp":"2018-12-09T05:23:09.561Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":16144,"fees":0}
{"height":26,"timestamp":"2018-12-09T05:23:17.595Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":89018,"fees":0}
{"height":27,"timestamp":"2018-12-09T05:23:42.667Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":100977,"fees":0}
{"height":28,"timestamp":"2018-12-09T05:24:00.747Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":72699,"fees":0}
{"height":29,"timestamp":"2018-12-09T05:24:18.804Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":72705,"fees":0}
{"height":30,"timestamp":"2018-12-09T05:24:39.881Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":32319,"fees":0}
{"height":31,"timestamp":"2018-12-09T05:24:53.919Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":111107,"fees":0}
{"height":32,"timestamp":"2018-12-09T05:25:16.017Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":88856,"fees":0}
{"height":33,"timestamp":"2018-12-09T05:25:38.100Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":19475,"fees":0}
{"height":34,"timestamp":"2018-12-09T05:25:57.128Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":146135,"fees":0}
{"height":35,"timestamp":"2018-12-09T05:26:15.261Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":24239,"fees":0}
{"height":36,"timestamp":"2018-12-09T05:26:34.284Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":125204,"fees":0}
{"height":37,"timestamp":"2018-12-09T05:26:48.377Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":44516,"fees":0}
{"height":38,"timestamp":"2018-12-09T05:27:04.442Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":78799,"fees":0}
{"height":39,"timestamp":"2018-12-09T05:27:14.504Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":14025,"fees":0}
{"height":40,"timestamp":"2018-12-09T05:27:38.517Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":0,"fees":0}
{"height":41,"timestamp":"2018-12-09T05:27:59.520Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":0,"fees":0}
{"height":42,"timestamp":"2018-12-09T05:28:17.522Z","harvester":"SCT6Y6-BMHHBM-JNA7LA-W6SYZX-ZSAUB5-K2EL7V-XEC2","txes":0,"fees":0}
  • [sec]:block production intervals
  • [TXs]:amount of transactions per block
  • [tx/sec]:transactions per second
Height Timestamp [sec] [TXs] [tx/sec]
1 2016-04-01T00:00:00.000Z 25 0000.00
2 2018-12-09T05:14:51.094Z 51 0 0000.00
3 2018-12-09T05:15:13.097Z 22 1 0000.05
4 2018-12-09T05:15:36.100Z 23 0 0000.00
5 2018-12-09T05:16:04.103Z 28 500 0017.86
6 2018-12-09T05:16:16.106Z 12 0 0000.00
7 2018-12-09T05:16:38.108Z 22 13146 0597.55
8 2018-12-09T05:17:06.123Z 28 113285 4045.89
9 2018-12-09T05:17:30.251Z 24 98982 4124.25
10 2018-12-09T05:17:56.340Z 26 105007 4038.73
11 2018-12-09T05:18:19.433Z 23 28303 1230.57
12 2018-12-09T05:18:41.470Z 22 153513 6977.86
13 2018-12-09T05:18:54.594Z 13 25990 1999.23
14 2018-12-09T05:19:19.624Z 25 127507 5100.28
15 2018-12-09T05:19:41.733Z 22 43008 1954.91
16 2018-12-09T05:20:05.772Z 24 140779 5865.79
17 2018-12-09T05:20:35.922Z 30 125231 4174.37
18 2018-12-09T05:20:47.040Z 12 28053 2337.75
19 2018-12-09T05:21:11.064Z 24 30522 1271.75
20 2018-12-09T05:21:35.092Z 24 179782 7490.92
21 2018-12-09T05:21:56.232Z 21 18894 0899.71
22 2018-12-09T05:22:14.254Z 18 138686 7704.78
23 2018-12-09T05:22:37.431Z 23 30691 1334.39
24 2018-12-09T05:22:51.469Z 14 118690 8477.86
25 2018-12-09T05:23:09.561Z 18 16144 0896.89
26 2018-12-09T05:23:17.595Z 8 89018 11127.25
27 2018-12-09T05:23:42.667Z 25 100977 4039.08
28 2018-12-09T05:24:00.747Z 18 72699 4038.83
29 2018-12-09T05:24:18.804Z 18 72705 4039.17
30 2018-12-09T05:24:39.881Z 21 32319 1539.00
31 2018-12-09T05:24:53.919Z 14 111107 7936.21
32 2018-12-09T05:25:16.017Z 23 88856 3863.30
33 2018-12-09T05:25:38.100Z 22 19475 0885.23
34 2018-12-09T05:25:57.128Z 19 146135 7691.32
35 2018-12-09T05:26:15.261Z 18 24239 1346.61
36 2018-12-09T05:26:34.284Z 19 125204 6589.68
37 2018-12-09T05:26:48.377Z 14 44516 3179.71
38 2018-12-09T05:27:04.442Z 16 78799 4924.94
39 2018-12-09T05:27:14.504Z 10 14025 1402.50
40 2018-12-09T05:27:38.517Z 24 0 0000.00
41 2018-12-09T05:27:59.520Z 21 0 0000.00
42 2018-12-09T05:28:17.522Z 18 0 0000.00

RESULTS

As we can see above, we achieved 4,000 transactions per second (!), by averaging 4,040 transactions per second using 8-38 blocks (2,529,116 transactions by 626 seconds).

…And how much did this cost?

Table contents listed respectively
VM SIZE, OFFERING, FAMILY, VCPUS, RAM (GB), DATA DISKS, MAX IOPS, TEMPORARY STORAGE, PREMIUM DISK SUPPORT, COST/MONTH (ESTIMATED).

This time we used the Azure VM F32_v2 that ran about 1,500 US dollars a month. This cost is excluding any costs for data transfers and disc usage.

Diagram pictured above shows a approximate price breakdown. A monthly cost of 175,655 Japanese Yen, usage costs and fees from network, discs and mijin.

Now I can finally tell people, in confidence, that 4,000 transactions per second is possible in a private Catapult environment. In addition, I did no tuning in this experiment so a more price point model could more than likely be possible. Please let me know what you think of anything above!

Thank you!
@planethouki

コメントを残す

メールアドレスが公開されることはありません。 * が付いている欄は必須項目です