Least fun Bouncy Castle ever. Making Java crypto play with everyone else.

This really sucked, and didn't appear anywhere on the web, so here what I found.

When you use Bouncy Castle to generate ECDSA (and probably the rest) signatures - it does something different that most - rather than just packing the r and s into a byte[] and calling it a day - it packs it into a DER structure, which hold both big integers in ASN.1.  Which is a bunch of acronyms and probably warns that this is going to be really annoying.  The spec appears to be for-pay only - so you need to look at source and guess what it means.  If you hex dump what BC generates:

00000000 30 46 02 21 00 D8 33 C8 A5 26 B5 99 16 D2 A0 2E 0F.!..3..&......
00000010 97 FC 1D 6B 96 EB FA 5A FF 8F 8A 19 8F 3E 56 83 ...k...Z.....>V.
00000020 40 79 8E DC 26 02 21 00 E7 E8 87 F8 4E 97 F3 43 @y..&.!.....N..C
00000030 79 7E D1 D8 E5 A9 BD C1 8C 70 2E BA 6D 9E CC 19 y~.......p..m...
00000040 83 ED 36 CE 65 05 82 BF                         ..6.e...

The packing is pretty easy - the first byte is always 0x30, second is total size, and the third is number of items.  After that it gets goofy - a big int is 32 bytes (0x20) - but the r and s blocks are sometimes 32 and sometimes 33.  It looks like for some reason if the high bit is set in the following element it throws in a random 0x00 to keep things from getting too crazy with all those ones.  If the size is 33 - toss the extra byte and read the next 32.  After which there's an element count to skip and another size for poor s.

Doing this by eye - here's the 'good' data from this hex dump:

00000000 30 46 02 21 00 D8 33 C8 A5 26 B5 99 16 D2 A0 2E 0F.!..3..&......
00000010 97 FC 1D 6B 96 EB FA 5A FF 8F 8A 19 8F 3E 56 83 ...k...Z.....>V.
00000020 40 79 8E DC 26 02 21 00 E7 E8 87 F8 4E 97 F3 43 @y..&.!.....N..C
00000030 79 7E D1 D8 E5 A9 BD C1 8C 70 2E BA 6D 9E CC 19 y~.......p..m...
00000040 83 ED 36 CE 65 05 82 BF                         ..6.e...

Back to Census library examples - really sending data!

Okay, I got distracted by Burning Man prep and left this sort of hanging - but back to using the Census library.  Just for the record, I don't own 'cathuggers.com' - please don't start blasting emails to people there.  Cats are awesome, get over it.

Last entry we were querying the blockchain using the REST API, which is cool enough but it's annoying for actually doing stuff.  I'm not a super fan of Python (the 2.x to 3.x breakage is a train wreck equal to the perl Parrot nonsense if you are old enough to remember that, my opinion as always) - but it's handy to do things.  To implement using the library is pretty easy...

__author__ = 'lpreimesberger'
import censusprotocollib
import json

print "Hello world!  I'm trying to query the blockchain server on this host!"wallet_address = raw_input( "which wallet to look for? : ")
print censusprotocollib.get_wallet_by_email("localhost", wallet_address )

Running this gets:



lpreimesberger@dadbuntu:~/projects/testthelibrary$ python testwalletquery.py 

Hello world!  I'm trying to query the blockchain server on this host!

which wallet to look for? : ilovecats@cathuggers.com

[{"_id":"55c3ffb0cdd86818fb3914b8","balance":0,"version":1,"created":1436734011857,"txid":"433225d2-e843-4448-a97d-aefaf4b5d5a0","item_type":"wallet","tag":"generated by censusprotocollib","source":"1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T","signature":"","block_in":"","email":"ilovecats@cathuggers.com","rsa_keys":["-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAseG7lt/JwOHyzG0hKbKQ\niDmiQ0bTY+mg6AQq8wdZx3lQF0BOWCtnYYxVL+zWr2fqMqjtsc6rC11p4VvmojBm\nGZAEMH1um+sUAzMLKellAdfRNH1XRA5Ag/treaWOlmY/2ilTfq2raSgq/Ed/kGix\nl+NynzgNiKqFVbBbCZ4rvPmkiyEmygpuLXU5MO8qOSpX1cegauByciKxcdqcBLjg\nQgj7XLO9eqGCX/ypzysXPAIgdWIUrZci8gjdJf65eQBywaQ1jLTpASV3kzY4pB4d\n3lBWZuIe5Ck65ZVQjeaPXgYVDLUFp+R2eERWAWydaKLqaSaL3NGg6XMuqoFzx0Ex\nrQIDAQAB\n-----END PUBLIC KEY-----"]}]







This built in encryption is using RSA RFC3447 - this is a deliberate choice to not use the ECDSA keys from Census so you can create and revoke without losing everything.  This also prevents easier cracking of your Census key, since there isn't any chance a lazy developer like me would implement something that would give predictable, encrypted messages that could limit the key possibilities.  RSA is probably good enough for most everything.




Doing this in code makes:




__author__ = 'lpreimesberger'
import censusprotocollib
import json

print "Hello world!  I'm trying to query the blockchain server on this host!"wallet_address = raw_input( "which wallet to look for? : ")
# a single email address can have many wallets - we'll cheat and use the first
the_wallet = json.loads(censusprotocollib.get_wallet_by_email("localhost", wallet_address ))[0]
print "Send this message on the ILOVECATS forum!"print "-------------------------------------------------------------------"print censusprotocollib.rsa_encode(the_wallet["rsa_keys"], "cats")
print "-------------------------------------------------------------------"







Running this will create a nice message for our cat loving friend:




lpreimesberger@dadbuntu:~/projects/testthelibrary$ python testwalletquery.py 

Hello world!  I'm trying to query the blockchain server on this host!
which wallet to look for? : ilovecats@cathuggers.com 

Send this message on the ILOVECATS forum! 

-------------------------------------------------------------------
L49L0Gt2tTNTv4IuHZRyF1wmCXz7NDrXtXev9jbTeADP8Uxeqy7COw9DlKKq+vOWrjnGb9it4prPBjObi7zptEi8J4I47Qb6FI8VHSbkNk3dei4Uev3pYY2BgQcFWJW4cQfkGohgBEPSGRlC7yXcmMw0uYXeekaAOzjkP7nTYUFV7XSGP8TpJGJCEwOLQLPRF0mmMkZiFXxuGTBew2CB9zqUUD10+hCtSsP61FJSVsf6JW2h1eNSsPWK1LE9mS5sfY+oYMckLrmI4jYQMlhiiVfgUk6DbgdjI6Scc420G6MkToOgZHL/tb/nVBeQefiojJPLpv2OOm6o9lKKXq4Xmg== 

-------------------------------------------------------------------





You can post this, or if you are super interested can use smtplib to send to a mail server.  You can also sign the message using your private RSA key using Crypto or use the censusprotocol.sign() function to sign with your Census key.  If you are sending stuff outside the protocol - you should probably only sign with your RSA key though.  Again, if you send predictable stuff - it's easier to guess your private key.  Keep the Census key for protocol packet - use RSA for other stuff.

Searching for Census Wallets

Okay, so the next step would be finding another wallet to do the operations on.  We know who we are, but how to find something else?

Quickest and easiest is by just hitting your local Census instance with the wallet id.  Census wallet id's are the same format as the usual bitcoin one.  The id number is a defined value that is a hash of the public wallet, which in turn is the hash of the larger private wallet key.  Early drafts of Census used variations of the basic Bitcoin wallet to differentiate it - but every one of these changes just made the network harder to manage and made less and less of the public code usable.

End result is - a Census wallet is almost the same as a Bitcoin wallet.   Most of the time, you could feed in your Bitcoin private key into the Census code and get a wallet with the same address for your digital items.  That's a terrible idea though - so don't please don't do that.  Generate a new key from some random data and add it.  Census is pretty secure - but if some vendor or online wallet is compromised you don't want your real money to disappear along with your digital items.  Spreading your assets out over more keys makes it less likely somebody can make away with all your stuff.

The URL to get a wallet by the wallet id is get a GET on the wallet route.  So this:

http://localhost:9000/api/v1/wallet/1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T

Would just barf out the wallet itself for that id:

[{"balance":0,"version":1,"created":1436734011857,"txid":"433225d2-e843-4448-a97d-aefaf4b5d5a0","item_type":"wallet","tag":"generated by censusprotocollib","source":"1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T","signature":"","block_in":"","email":"ilovecats@cathuggers.com","rsa_keys":["-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAseG7lt/JwOHyzG0hKbKQ\niDmiQ0bTY+mg6AQq8wdZx3lQF0BOWCtnYYxVL+zWr2fqMqjtsc6rC11p4VvmojBm\nGZAEMH1um+sUAzMLKellAdfRNH1XRA5Ag/treaWOlmY/2ilTfq2raSgq/Ed/kGix\nl+NynzgNiKqFVbBbCZ4rvPmkiyEmygpuLXU5MO8qOSpX1cegauByciKxcdqcBLjg\nQgj7XLO9eqGCX/ypzysXPAIgdWIUrZci8gjdJf65eQBywaQ1jLTpASV3kzY4pB4d\n3lBWZuIe5Ck65ZVQjeaPXgYVDLUFp+R2eERWAWydaKLqaSaL3NGg6XMuqoFzx0Ex\nrQIDAQAB\n-----END PUBLIC KEY-----"]}]

Wallets are case sensitive though - 1Jw is not the same as 1JW, so don't normalize the data.

Census is not as privacy mad as Bitcoin and most of the altcoins - people can't find or remember wallet addresses.  They are just too many digits and the format is sort of weird.  People can remember phone numbers and email addresses - but it's really unreasonable to think your friend can remember that you are 1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T and send you something.  You don't need to add an email address - but if you do you can make life easier.

http://localhost:9000/api/v1/wallet/email/ilovecats@cathuggers.com

Will return the wallet that matches that address:

[{"balance":0,"version":1,"created":1436734011857,"txid":"433225d2-e843-4448-a97d-aefaf4b5d5a0","item_type":"wallet","tag":"generated by censusprotocollib","source":"1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T","signature":"","block_in":"","email":"ilovecats@cathuggers.com","rsa_keys":["-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAseG7lt/JwOHyzG0hKbKQ\niDmiQ0bTY+mg6AQq8wdZx3lQF0BOWCtnYYxVL+zWr2fqMqjtsc6rC11p4VvmojBm\nGZAEMH1um+sUAzMLKellAdfRNH1XRA5Ag/treaWOlmY/2ilTfq2raSgq/Ed/kGix\nl+NynzgNiKqFVbBbCZ4rvPmkiyEmygpuLXU5MO8qOSpX1cegauByciKxcdqcBLjg\nQgj7XLO9eqGCX/ypzysXPAIgdWIUrZci8gjdJf65eQBywaQ1jLTpASV3kzY4pB4d\n3lBWZuIe5Ck65ZVQjeaPXgYVDLUFp+R2eERWAWydaKLqaSaL3NGg6XMuqoFzx0Ex\nrQIDAQAB\n-----END PUBLIC KEY-----"]}]


Easy enough right?  Now that we have a public id - we can send stuff pretty easy.

Adding Census Protocol wallets to the network

So now that we have a wallet – that's pretty cool, but adding to the network needs some extra work.

We can send anything we want (within reason) to the network to add to the wallet. The network will blindly send whatever your want – so if you do something really dumb like send your own private key to the network it will happily share it to every person on Earth as the data is replicated. The Python library has a handy function call that removes the private data from the common CWIF format - cwif_to_wallet.

Using this function:

__author__ = 'lpreimesberger'

import  os

import censusprotocollib

print "Let's make a wallet!"

email = raw_input( "What is your email address? : " )

fake_seed = raw_input( "Enter something for a brainwallet: " )

new_wallet = censusprotocollib.generate_wallet(fake_seed, email)

cp_wallet = censusprotocollib.cwif_to_wallet(new_wallet)

print cp_wallet

Which outputs:

$ python createwallet.py

Let's make a wallet!

What is your email address? : ilovecats@cathuggers.com

Enter something for a brainwallet: correct horse battery staple

{'balance': 0, 'version': 1, 'created': 1436734011857, 'rsa_keys': ['-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAseG7lt/JwOHyzG0hKbKQ\niDmiQ0bTY+mg6AQq8wdZx3lQF0BOWCtnYYxVL+zWr2fqMqjtsc6rC11p4VvmojBm\nGZAEMH1um+sUAzMLKellAdfRNH1XRA5Ag/treaWOlmY/2ilTfq2raSgq/Ed/kGix\nl+NynzgNiKqFVbBbCZ4rvPmkiyEmygpuLXU5MO8qOSpX1cegauByciKxcdqcBLjg\nQgj7XLO9eqGCX/ypzysXPAIgdWIUrZci8gjdJf65eQBywaQ1jLTpASV3kzY4pB4d\n3lBWZuIe5Ck65ZVQjeaPXgYVDLUFp+R2eERWAWydaKLqaSaL3NGg6XMuqoFzx0Ex\nrQIDAQAB\n-----END PUBLIC KEY-----'], 'txid': '433225d2-e843-4448-a97d-aefaf4b5d5a0', 'item_type': 'wallet', 'tag': 'generated by censusprotocollib', 'source': '1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T', 'signature': '', 'block_in': '', 'email': 'ilovecats@cathuggers.com'}

Which is all public key data and is safe to send out. The actually call to submit the new wallet to the network is .

Making a bunch of changes to save the local CWIF and the to send the data we get this:

__author__ = 'lpreimesberger'

import os

import json

import censusprotocollib

print "Let's make a wallet!"

email = raw_input( "What is your email address? : ")

fake_seed = raw_input( "Enter something for a brainwallet: ")

name_of_cwif = raw_input( "Enter name of CWIF file: ")

new_wallet = censusprotocollib.generate_wallet(fake_seed, email)

cp_wallet = censusprotocollib.cwif_to_wallet(new_wallet)

f = open( name_of_cwif + ".cwif", "w " )

f.write( json.dumps(new_wallet) )

f.close() 

signed    = censusprotocollib.sign(cp_wallet, new_wallet["private"])

print censusprotocollib.wallet("localhost", signed )

Which will generate, clear, and submit the wallet to the local network. It doesn't become real until the server processes it – but more on that soon.

Being able to access a wallet balance and info from the API is pretty cool – but the next obvious step is to create your own wallet or owned wallet out there to hold stuff. One of the things that you'll notice is that the wallet query itself just returns the current balance – not what's owned. Finding stuff that's is owned isn't super difficult – but it's a different query and is more expensive than what that entry is intented for. Much of Census occurs on a server elsewhere unless you are running your own full node at home – so we don't return and do stuff that we don't need or want.

The entry point in the library to generate a new wallet is generate_wallet

– it's pretty expensive to run in terms of CPU time, which again is deliberate since this makes is more difficult to spam the network with junk data. Generate wallet returns a format which is being called CWIF – or Census Wallet Interchange Format. What it outputs, you should not post to the network or anything like Dropbox or Drive. All those items are indexed, and the private key inside the CWIF can give access to all the items in your wallet.

We'll make an app that creates what's lovingly called a 'brainwallet' – which is a wallet generated from a simple string. The most common example is the ever popular horse battery password from xkcd.com:

Real world, you probably shouldn't use these. This is awesome for passwords, but for generating random data, you are better off with random data – not using Monty Python quotes. We don't care for this example though – just use one of the popular entropy generators to create a nice, random, 2048 byte block to make your keys. Now the code:

__author__= 'lpreimesberger'

import os

import censusprotocollib

print "Let's make a wallet!"

email = raw_input( "What is your email address? : " )

fake_seed = raw_input( "Enter something for a brainwallet: " )

new_wallet = censusprotocollib.generate_wallet(fake_seed, email)

print new_wallet

Running this:

$ python createwallet.py

Let's make a wallet!

What is your email address? : ilovecats@cathuggers.com

Enter something for a brainwallet: correct horse battery staple

{'rsa_keys': [{'public': '-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2Xq+nsbKILYaaYIyDB8X\n6jjsvTm+Z00yaiy5HxxIABuTx6gP3DRE78G0Wn5yfw7HRNDYEUaxwnNVFXFwuC+g\nv+uAIdrzXLVO/Qp3Hx7Y1+DoqRtsmG+w05XYY3GrWWT5u7lB93yFTQsOlqBORic2\nSy0Uv6jh9PbMJsJihDPUEsnXYHupBNINVXHaqX6skFsGwlkmtLtEcDGVx22JxTOG\noNrRURMZxaTIYzLYosu0/kl7VgnYFcvR3wa0czUkYDsDr0yWy8zJJNBlDTnx+cNf\naIV1aMmgHclB7ruGRAW4ajPR+cNzWDfns7P440rSxGaEg/T5YwhfgtwSScU/HCED\nPQIDAQAB\n-----END PUBLIC KEY-----', 'private': '-----BEGIN RSA PRIVATE KEY-----\nMIIEpQIBAAKCAQEA2Xq+nsbKILYaaYIyDB8X6jjsvTm+Z00yaiy5HxxIABuTx6gP\n3DRE78G0Wn5yfw7HRNDYEUaxwnNVFXFwuC+gv+uAIdrzXLVO/Qp3Hx7Y1+DoqRts\nmG+w05XYY3GrWWT5u7lB93yFTQsOlqBORic2Sy0Uv6jh9PbMJsJihDPUEsnXYHup\nBNINVXHaqX6skFsGwlkmtLtEcDGVx22JxTOGoNrRURMZxaTIYzLYosu0/kl7VgnY\nFcvR3wa0czUkYDsDr0yWy8zJJNBlDTnx+cNfaIV1aMmgHclB7ruGRAW4ajPR+cNz\nWDfns7P440rSxGaEg/T5YwhfgtwSScU/HCEDPQIDAQABAoIBAQCMmgCXIioXj+Pv\nho7Yq1mIwhi8FZL1skD/x74/UMdz2mBmWk82rExNPUxenRVk86b85JBsSuotrua5\nIEghrBkfOx/xw15G96kDizqMUiMbHnsMqYBB5SPLBzuIvzpqZw/Vv+XYHRcJQjbC\nRw2T6VH+tDyP1J2RA+XkwRN0KRXYtphIHJkIalahco2VAkXFmW5nCD8jV7s5BC0t\naJtKOiigI1TVB2RicDwFHsV23IwjGu3kuD7ZP4mcJYfTMNRPuy9TOrPuihgXtf3q\n5H/bB8k0g7t13tCNJWcnOBkjYMXAV/y3PfeB6XJibs8b+L7f9cmhuOX+7tvvmT1l\nKkBiDCvBAoGBAOMbBrhIoZRxEYbF5KNVkclHD+kDR7HzacfFV8X2DivIrfnTNIGB\n7bMMiwq5qqE0pUAsuTPIZ1ObhhWCSsa5uhlgeVlxFe3wt/7wg0P0E1Tu73xmEkLm\ny2RpR4kl/ZYFzZxSL+24nP7rEbqoTmjMBJM4KGBxyYX458rD7RqhbsKlAoGBAPUm\nMKu6rsGK4w75/oi9snY2mLqJIkeS9VL1hNlcrWRBj2e/oSB9HEvdkZ1SsfVsMHjj\ngTjg4C4Zpgh9mxXneGbX5OAVFgOdF8XTCOgykePN4/BfeLfPF/DaGy5c5LWRnobd\nJxZnIXaN38Dm/adK2h7hRn2b/NFCMTN1/gEkz9K5AoGBAIN9rWlbnDovMR+gSp4L\nOEqam2qlXIh6z987snNK4K+XFKwWDkuBp9fIRmTiVRnbOmaqvlxuBu1QmdSwSI1B\nEbAJvJj9sd2/RzkTllzs35iIcx7Kln4n1NGBZ0rQNT/8giylJVz1S+kIv44cTrG8\nllWK6U3V9AFeuQ4cXJbTqyaFAoGBAINXOqxHEGnRJ2VWuvA38e0zww/teNgv/A++\nSrbJ2HvmRBaprIHjhc1/oAxRydZbaHzcJtd6Y6FPTI0v/m713KP89zq3EVxORfZO\n4m7E6TTLkluuStbvtTvtHrLwQfilU/Gdetud1WfXCxImr695gvMYbtPNRt42QHX5\nuWSTfGTBAoGAAY98CTF3ltSxWmiZSFKCm1VqbU4B1EfU87K8zXcfY4Z4+90A64ox\nHDSjvJXe6nZ3xmH2XoUnGA8Eczqi+YN8wiCUCxImq2diUKgeXTKqUL2c1Uw9Xnwq\nmKyeGdE6Ie3OPQdTjRQEH7yJjsLoSqsWxPv9kZrkqoQhERJWaaT68Xg=\n-----END RSA PRIVATE KEY-----'}], 'address': '1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T', 'public': '0478d430274f8c5ec1321338151e9f27f4c676a008bdf8638d07c0b6be9ab35c71a1518063243acd4dfe96b66e3f2ec8013c8e072cd09b3834a19f81f659cc3455', 'private': 'c4bbcb1fbec99d65bf59d85c8cb62ee2db963f0fe106f483d9afa73bd4e39a8a', 'email': 'ilovecats@cathuggers.com'}

Holy crap! Yeah, that's a lot – and it includes some of the secret data we shouldn't put on the Internet. Running that through a pretty printer and clipping the data:

{

'rsa_keys': [

{

'public': '-----BEGINPUBLICKEY...

'private': '-----BEGINRSAPRIVATEKEY...

}

],

'address': '1JwSSubhmg6iPtRjtyqhUYYH7bZg3Lfy1T',

'public': '0478d4...

'private': 'c4bbc...

'email': 'ilovecats@cathuggers.com'

}

In this output:

• rsa_keys are the set of RSA interchange keys that are associated with this wallet – it's an array, so there can be a lot of these. Only public keys go on the network
• address is the wallet address, which is actually the unique hash of the private key if you follow it all the way back
• public is the public key – which goes to the network as a backup (many clients can validate based on address, but more info doesn't hurt)
• private is the private ECDSA key we keep locally and don't share
• email is the optional email tag which lets folks find us on the network. This is optional and can be ignored if you are feeling paranoid or it doesn't make sense. For brokers, this could be a randomized email for your internal customers (i.e. - 2342356134@ticketbrokers.com) to simplify transactions

We can't do anything with this other than save it locally though – so what about adding wallets to the network?