Category: enterprise

invite_me: Onboarding to Ethereum chains with keybase.io identity verification and EIP-712

This is not a Blockchain-bullshit-only post. Head to https://github.com/ice09/onboarding-eip712 for a sample implementation of a private Testnet onboarding with Twitter, Reddit or Github Keybase proofs.

The challenge…

Creating a private Ethereum Proof-of-Authority testnet is actually quite easy, just follow the instructions of Parity or, for testing purposes only, cliquebait, and you’re done.

But how do you onboard new users, who want to participate in your testnet, if they don’t have any ETH to start with and you still want to stay as close to your production environment in terms of gas usage as possible and therefore don’t want to enable gasless transactions in your PoA testnet?

…and the public testnet solutions

Kovan PoA

The public (Parity-powered) Kovan PoA solves this problem by providing a “Faucet Service” which is verified by

Rinkeby PoA

The public (Geth-powered) Rinkeby PoA solves this problem by providing a “Faucet Service” which is verified by

However, these methods require trust or establish dependencies on the testnet providers, which you will and can most likely not introduce in your private net. 

Analysing the problem

There are (at least) two different user states which a user can have in your PoA private testnet:

  1. The user has an authorized account in one of the authority node wallets.
  2. The user is connected to the private testnet, but initially has no balance.

In Proof-of-Work (PoW), the non-authorized user could start mining and get ETH for contract development. This is the Ropsten solution. In PoA, mining is not possible, so the user has no possibility to transact on the chain, except for using usd_per_tx=0 or similar for no gas costs on the authority nodes, which is possible, but contradicts the security measures established by having gas usage for transactions to minimize code execution, prevent infinite loops, etc. Even more, the setup on a test stage should always be similar to the production environment.

Recentralizing for “Almost Know Your Customer”

keybase.io does a great job in identifying users without revealing their real life identity if they don’t want to. But they make it quite difficult for attackers to create several identities. So a good enough solution for onboarding otherwise unknown users could be to bind their testnet accounts to their keybase users.

The basic idea and the following approach is inspired by Aragon and one of their great blog posts, which every Ethereum/Solidity dev should take a look at.

In our setup, we changed two crucial factors:

  1. removed dependencies on a product (Aragon) and on the Oracle (oraclize.it) and
  2. introduced a middleware component as a new dependency, but which you control and can (and have to) host on your own server.

keybase allows for publishing files per HTTPS by copying them to KBFS file system. Thereby, someone reading a file from https://KEYBASE_USER.keybase.pub/invite_me.json can be sure that the file invite_me.json was stored by keybase KEYBASE_USER.

But this is just one proof, how can we be sure that the user really has the private key for the address he wants to be registered? We can just sign the message with the private key and ecrecover in a contract. 

Using this mechanism, we can be sure that:

  1. If the URL https://KEYBASE_USER.keybase.pub/invite_me.json gives back the correct JSON, the user is who he pretends to be, since he had to copy invite_me.json to KBFS
  2. The address (private key) belongs to the user, since he signed the message in invite_me.json and the signature is ecrecovered in the invite_me contract.

MetaMask does a great job in helping signing data with your Ethereum’s account private key. However, up to EIP-712 the signed data has been displayed as a non-human-readable Hex string.

This is unacceptable, especially in crypto, a field of very easy and unavoidable fraud attempts. 

Therefore, we are supporting EIP-712 by signing with a MetaMask version supporting the new eth_signTypedData method. EIP-712 is a major step forward and should be the only way users are required to sign data with their Ethereum account’s private key.

The following overview shows the process which let a new user register his keybase user to a new generated address. Afterwards, 10 ETH are sent to this address. The user can only register once. However, the contract owner can manually unregister users if necessary.

For the ETH transfer to happen, the user must

  • have the invite_me.json as generated by the frontend part stored in his KBFS public directory, so that it can be retrieved by the server side process at the keybase.pub domain
  • have at least one of these Keybase proofs: Twitter OR Github OR Reddit

Introducing invite_me

The complete process is implemented in the “DApp” invite_me, which consists of a JavaScript frontend and a web3j-powered Java backend component called verifier.

After completing the steps, you can see the 10 ETH loaded into your account in MetaMask.

Let me try this!

First, checkout this. Then, make sure that you have an understanding how the authorization and verification works and what the Java component verifier does and how they depend on each other.

Last, install it locally and try it out. And, most importantly, please comment here or in the reddit post if it doesn’t work for you or if you have remarks about the approach, obviously this is work in progress.

What comes next?

We’d like to try two different approaches:

  • Using 3box as a more decentralized (but not as mature) keybase alternative
  • Realizing a different use case: “Almost KYC Airdrops” 

Excel2Blockchain with web3j

If you ever come to the conclusion, for whatever reason, that pushing Excel data to the Blockchain is what you need, we are happy to help!

We recently read about the Azure Blockchain Development Kit and had to cringe about the samples. But, who are we to judge those important enterprise business use cases requiring to get data from Excel to the Blockchain? 

Therefore, we took a look at the prerequisites, setup, initialisations and finally the execution of this with the new Microsoft Azure templates. And, to be clear, we love the idea of Microsoft getting involved into Blockchains and Ethereum and pushing Azure as a starter kit, that’s really great.

However, the whole procedure seemed quite long and exhaustive, that’s why we want to show how easy this is with standard development methods. Of course, these are lacking a lot of features and cool stuff you get with Azure then. But, they can help you to understand better what happens under the hood. So, let’s go for it.

No, wait, Excel to Blockchain, honestly? Why?

//TODO: Insert suitable reason here.

Ok then, but how?

Due to web3j it is really easy to connect to an arbitrary Ethereum Node, even Infura and Ganache, as simple as it is with web3.js or web3.py. All the other stuff is common Java dev tools, like Glueing with Maven, Excelling with Apache POI, etc. This is where Java shines, it’s really good at enterprisey integration stuff.

You can start directly and import the sources as a Maven project in your IDE or build it without an IDE and start it from the command line. See the details here.

Show me some code!

private void connectToBlockchain() throws IOException {
    Web3j httpWeb3 = Web3j.build(new HttpService("http://localhost:8545"));
    log.info("Connected to HTTP/JSON Ethereum client version: " + httpWeb3.web3ClientVersion().send().getWeb3ClientVersion());
    // Create credentials from private key. 
    // Don't use with real (Mainnet) Ethereum accounts.
    String privateKeyDeployer = "c87509a1c067bbde78beb793e6fa76530b6382a4c0241e5e4a9ec0a0f44dc0d3";
    Credentials credIdentity = Credentials.create(pkDeployer);
    BigInteger balance = httpWeb3.ethGetBalance(credIdentity.getAddress(), DefaultBlockParameterName.LATEST).send().getBalance();
    log.info("Deployer address " + credIdentity.getAddress() + " has " + balance + " wei.");
}

private void deployExcelContractToBlockchain() throws Exception {
    ExcelStorage excelStorageContractHttp = ExcelStorage.deploy(httpWeb3, credIdentity, ManagedTransaction.GAS_PRICE, Contract.GAS_LIMIT).send();
    log.info("Deployed contract at " + excelStorageContractHttp.getContractAddress());
}

As you can see, ExcelStorage is a class which can used like any other Java class. It has been generated by a Maven plugin (web3j-maven-plugin) during build time. It is statically typed with methods resembling the Solidity functions of the smart contract and with all the nice aspects this has: compile time checks instead of runtime checks and brilliant IDE support (code completion, type checking, static code analysis).

With great IDE and build tool support, you can try this out yourself, you will be surprised how lightweight and easy Ethereum smart contract usage is from Java. Even more, integration with your existing (“legacy”) software and infrastructure is straightforward. 

Have fun!

Addendum: This is a Testsetup only! What about the real Blockchain?

Where does the private key in the current Java sources come from? 
If you start Ganache with the mnemonic mentioned, everytime the same accounts are generated. The first one being 0x627306090abaB3A6e1400e9345bC60c78a8BEf57. This address has one private key which is set into the Java sources to create the Credentials for contract deployment and transactions.

Setup for other test chains

  • Create an account with infura.io
  • Use the Endpoint URL from the dashboard and copy in into the Java sources

  • Create an account with MetaMask
  • Copy the address of your account

  • Copy the private key to the Java source code to use the account for contract deployment and transactions

Blockchain + Streaming Analytics = Smart Distributed Applications

We are really pleased to publish a guest contribution by Kai Wähner about Smart Distributed Applications.
Kai is Technology Evangelist and Community Director for TIBCO Software. His expertise lies within the fields of Big Data, Advanced Analytics, Machine Learning, Integration, SOA, Microservices, BPM, Cloud, Internet of Things, Blockchain and Programming Languages such as Java EE, Scala, Groovy, Go or R. He regularly writes about new technologies, articles and conference talks on his blog.

We are approaching Blockchain 3.0 these days. You ask “3.0”? Seriously? Blockchain is still in the early adoption phase! Yes, that is true. Nevertheless, while we are moving forward to real world use cases in various industries, we see that you need more than just a blockchain infrastructure. You need to build decentralized applications. These include blockchains, smart contracts plus other applications plus integration and analytics on top of these solutions.

Middleware is Key for Success in Blockchain Projects

Blockchain is the next big thing for middleware! There is no question around this. You need to interconnect other applications, microservices and cloud offerings with a blockchain infrastructure to get real value out of it. In addition, visual analytics and machine learning have to be leveraged to find insights and patterns in blockchain and non-blockchain data. Finally, streaming analytics is used to apply these insights and patterns to new events in a blockchain infrastructure. There is a variety of use cases like fraud detection, compliance issues, optimization of manufacturing or supply chain processes, or any kind of scenarios with the Internet of Things (IoT).

Reference Architecture for Blockchain and Middleware

Variety of Blockchain Platforms including Hyperledger and Ethereum

The blockchain market is growing significantly these days. You need to think about various blockchain characteristics for your next blockchain project.

  • Who are the users of the blockchain? Is it public or private? Which partners do you need to work with?
  • Do you want to build your own blockchain infrastructure (based on a framework such as Hyperledger and one if its implementations like IBM’s Hyperledger Fabric, Iroha, Intel’s Sawtooth Lake) or leverage an existing platform (like Ethereum)?
  • Do you want to host the infrastructure yourself or leverage a cloud service like IBM’s Bluemix with focus on Hyperledger or Microsoft’ Azure with focus on Ethereum?
  • What development environment and tooling do you want to use (like Truffle or BlockApps on top of Ethereum)
  • Which characteristics are important for your scenario? What about speed, security, consensus algorithms, integration of non-blockchain services, and other important aspects? Maybe an add-on on top of a blockchain is needed, like the Raiden Network to leverage off-chain state networks to extend Ethereum with some nice properties like scalability or high performance for asset transfers?
  • Or do you want to focus on a industry-specific blockchain solution like R3 Corda or Ripple for financial services?
  • What middleware do you need? Do you need Application Integration or API Management to interconnect everything? Visual Analytics to find insights and patterns in historical blockchain data? Streaming Analytics to apply rules to action in (near) real time for new blockchain events?

The following shows how to leverage Streaming Analytics together with blockchain events. This example uses TIBCO StreamBase in conjunction with the public Ethereum test network. Note that similar scenarios can be build with any other blockchain infrastructure. A follow-up post about how to leverage middleware with Hyperledger will come soon, too.

Streaming Analytics for Correlation of Blockchain and Non-Blockchain Events

The scenario uses a Smart Contract to define a Coin system. You can mine coins and transfer them to other users (i.e. blockchain addresses). This example is similar to Bitcoin concepts to show how to leverage streaming analytics with any custom blockchain application and smart contracts. The goal is not to show the power of smart contracts (other articles are available for this). The programming language used to develop this Smart Contract is Solidity; more or less the de facto standard to write smart contracts for Ethereum.

Here is the Smart Contract built and deployed with Browser Solidity:

Smart Contract ‘Coin’ developed and deployed with Browser Solidity

MetaMask, a bridge to run Ethereum dApps in your Chrome browser, is running in the background to connect to the Ethereum network and commit the transactions developed with Browser Solidity. You could also use Streaming Analytics to deploy smart contracts, of course. However, in this example TIBCO StreamBase was only used for the following two parts:

  • Receive new events from the blockchain network: You can filter, aggregate, analyse or transform any events like pending transactions, logs or blockchain blocks – and also combine this information with non-blockchain events, of course. For example, you could build a streaming analytics process to analyze just the logs relevant for your specific transaction IDs to spot issues and act proactively, let’s say if a pending transaction takes too long or fails.
  • Execute transactions on the blockchain network via smart contracts: You can mine new coins, send coins to other blockchain addresses and also check the balance of an address. Anything what the smart contract allows can be included into the streaming analytics process.

The streaming analytics process monitors all Ethereum events continuously. This is not as trivial as you might know it from classical messaging systems. You cannot just listen to a topic or queue, but you have to pull information out of the blockchain. Depending on the use case, you have to implement some solution which solves your problem but also does not consume too many resources. This is always a trade-off, which has to be thought through when building your streaming analytics process. This also highly depends on the blockchain infrastructure you use and its feature set.

Please note that security considerations are not part of this example. In the real world, you would integrate encryption and other security requirements into the streaming process, of course. In this demo, we use “hardcoded” private keys for sending transactions. A no-go in a real world project.

Let’s now take a look at an implementation of this process.

TIBCO StreamBase + Ethereum Blockchain

Here is the demo setup:

Architecture with TIBCO StreamBase + Ethereum Blockchain

The Ethereum test network is a distributed peer to peer ledger. It runs on various Ethereum clients. We used one of the most mature ones on our local laptop: The geth client implement in Golang. This is synced and also part of the Ethereum test network.

TIBCO StreamBase is used to build the streaming analytics process:

TIBCO StreamBase Connectors for Ethereum

The web3j Java API is used to connect TIBCO StreamBase with the Ethereum network through our local geth client. You just need to write the connector once and can reuse it in all your streaming processes via drag&drop and configuration. These behave “just” like any other connector (such as messaging via MQTT or Apache Kafka)  and components to build streaming logic (like filter, aggregate or transform).

For more details, please check out my live demo of combining streaming analytics and Ethereum blockchain

Building this process was actually a pretty easy task with TIBCO StreamBase. In the same way, you can build much more sophisticated blockchain processes in your real world project. Let’s also think about some other next steps.

Next Steps: Application Integration, API Management, Machine Learning, and more

A real world blockchain projects needs streaming analytics to correlate blockchain and non-blockchain events to fight fraud or compliance issues, to improve efficiency in manufacturing or supply chain processes, to combine Internet of Things with blockchain infrastructures, and for many other use cases.

Though, Streaming Analytics is just one piece of the puzzle. Here are some more thoughts about why you might combine blockchain with middlware and analytics:

  • Live Visualization for Real Time Monitoring and Proactive Actions
  • Cross-Integration with Ethereum and Hyperledger Blockchains
  • Data Discovery for Historical Analysis to Find Insights and Patterns
  • Machine Learning to Build of Analytic Models
  • Application Integration with other Applications (Legacy, Cloud Services, …)
  • API Management to expose blockchain services and handle caching / throttling challenges
  • Native Hardware Integration with Internet of Things Devices

I will do more posts about these ideas and show more live demos in the next weeks and months. In the meantime, first customer projects also kicked off, already. Blockchain and middleware have a great and interesting future…

Keywords:

Blockchain, Ethereum, Hyperledger, Middleware, Integration, Streaming Analytics, TIBCO, StreamBase, Live Datamart, Smart Contracts, Cloud, web3j

Static Type Safety for DApps without JavaScript

DApps, starting professionally…

You might not be aware, but despite its similarities to JavaScript, Solidity is actually a statically, strongly typed language, more similar to Java than to JavaScript.

solidity
static type check in browsersolidity

…and ending in frontend-chaos

Sadly, for a long time, there has only be one interface to Ethereum nodes, web3.js (besides JSON/RPC), which is, as its name implies, written in JavaScript.

Though providing this API in a web-native language is really a brilliant idea in terms of fast development, seperation of concerns and ease of use, it is a nightmare for professional, multi-developer, multi-year, enterprise products.

You may not agree with me here, but as there are currently no 10 year old 1.000.000 LoC enterprise projects in node.js/JavaScript out there, you should at least consider that such projects are nearly impossible to maintain with a dynamically, weakly typed language like JavaScript (JS).

So, we have this situation, where JS defines the lowest common denominator (dynamically, weakly typed) 

JavaScript_1 (2)

when we really would like to have this situation, where Java (C#, Haskell) defines the lowest common denominator (statically, strongly typed)

JavaScript_2 (2)

Removing chaos

The problem is was, that up to now only web3.js existed. However, today there is also a web3.py (which is Python and therefore at least strongly typed, but still dynamically) and, brandnew, web3j.

With the latter, we can easily model the call chain above, where we only use statically, strongly typed Java and omit JavaScript altogether. Welcome to hassle-free integration into existing Java/JEE-environments without workarounds. Finally: using the Ethereum Blockchain with Java.

If you want to actually get deeper and use Java with no RPC at all, you can also switch to EthereumJ, which is a Ethereum Node implemented in Java, like Eth (C++), Geth (Go), PyEthApp (Python) or Parity (Rust). It is crucial to understand the difference between web3j and EthereumJ. If you just want to use some Ethereum Node from a Java application, web3j is your choice, you are limited to the Web DApp API then, which should be enough for all “Ethereum user” use cases.

We will not explain in detail how to use web3j, it should be familiar to any Java developer how this library can be used just by adding Maven-dependencies to your project.

Fixing the front-end

We could stop here, since using JavaScript for the frontend is not really problematic and a common use today.

However, if you use JavaScript in your frontend, it might really make more sense to stay with web3.js. So, we want to go further: how are we going to create the GUI if we want to have no JavaScript at all?

This is just a PoC, but if you think of any other client to the Ethereum Blockchain other than a web site (let’s say: Batchjobs, Web Services, Message Queues, Databases, other proprietary software with Java adapters (there are some!)), this should make sense to you – you really wouldn’t want to use them from web3.js (hopefully).

Using templates: Thymeleaf and Spring Boot for slim enterprisy software

We will do a step-by-step guide for creating a No-JS-Dapp. Even without any Java experience, you will be able to follow without problems. Java is not that complicated anymore!

  • Get an infura.io account and key, so you don’t have to mess around with starting your own Ethereum node
  • Clone this repo: https://hellokoding.com/spring-boot-hello-world-example-with-thymeleaf/
  • Install Maven
  • Edit these files:

    pom.xml (add these dependency to section dependencies and add the repo, beware that web3j is a fast moving target, check for new versions)

    <dependency>
        <groupId>org.web3j</groupId>
        <artifactId>core</artifactId>
        <version>0.2.0</version>
    </dependency>

 

<repositories>
    <repository>
        <id>oss.jfrog.org</id>
        <url>http://dl.bintray.com/web3j/maven</url>
    </repository>
</repositories>

src/main/resources/templates/hello.html (change name to balance.html)

<!DOCTYPE html>
<html lang="en" xmlns:th="http://www.thymeleaf.org">
<head>
 <meta charset="UTF-8"/>
 <title>Your Static Strongly Typed Wallet</title>
</head>
<body>
 <p th:text="'The balance of account ' + ${address} + ' is ' + ${balance}" />
</body>
</html>

src/main/java/com/hellokoding/springboot/HelloController.java (change name to EthereumController.java)

@Controller
public class EthereumController {

    @RequestMapping("/balance")
    public String balance(Model model, @RequestParam(value="address", required=false, defaultValue="0xe1f0a3D696031E1F8ae7823581BB38C600aFF2BE") String address) throws IOException {
        Web3j web3 = Web3j.build(new HttpService("https://consensysnet.infura.io/{YOUR_INFURA_KEY}"));
        EthGetBalance web3ClientVersion = web3.ethGetBalance(address, DefaultBlockParameter.valueOf("latest")).send();
        String balance = web3ClientVersion.getBalance().toString();
        model.addAttribute("address", address);
        model.addAttribute("balance", balance);
        return "balance";
    }

}

…that’s it. Start with mvn spring-boot:run

If you encounter an connection/handshake error, you may have to import the infura certificate into your local Java keystore (I didn’t have to)

$JAVA_HOME/Contents/Home/jre/bin/keytool -import -noprompt -trustcacerts -alias morden.infura.io -file ~/Downloads/morden.infura.io -keystore $JAVA_HOME/Contents/Home/jre/lib/security/cacerts -storepass changeit

Look Ma! Displaying the wallet balance with no JavaScript!

You can call the spring-boot web application with http://localhost:8080/balance (then the defined default argument is used) or with your address (in the consensys testnet) as parameter address= 

walletOf course, you can change the Ethereum net like you want in file EthereumController to morden or mainnet, just read the welcome mail from infura.io. Or you can just use a local Ethereum node like geth with RPC enabled (geth –rpc) and http://localhost:8545 as the constructor for HttpService of the Web3j-Factory in EthereumController.

Have fun, with or without JavaScript!