Month: August 2018

PRJ701 -26/08/2018 – [insert great blog title here]

Another week down and another round of progress towards the end goals. This week has been eventful, I had my first meeting with Sandra and participated in the first seminar session, as well as made some more progress in terms of the development of the project artifact and contemplating a decision around the virtual environment portion of my project.

Meeting with Sandra

As mentioned above, I had my first meeting with Sandra, in this meeting we discussed my overall project and my main objectives for the project, we also discussed the potential implications of my project being taking from a prototype into production on a wide scale, which was an exciting prospect for us both.

We also briefed over the potential change I am considering (more on this below) and I demonstrated what I had achieved so far.

After my meeting with Sandra I feel even more excited to work on this project and feel extremely confident that she will be a fantastic supervisor overall. Further to this, having the additional support of her advanced Accounting knowledge to support my fairly basic and limited existing knowledge fills me with even more confidence.

Another Potential Change to my Original Proposal

I stated in my previous blog that my main goal and focus for this week would be to set up the virtual environment and deploying the Blockchain across the multiple hosts.

My initial plan was to run multiple instances of Ubuntu within different subnets and connect them with one (or more) instance(s) of VyOS, however, after my discussion with Sandra I have started contemplating a change in my plan to remove the virtual environment component from the overall scope of my project.

The primary reason I am considering dropping this component of my project is because I am unsure if it will really add any great benefit towards the actual end goal of my project, my reason for including this component as part of my project in the first place was purely so that my project made strong use of all the knowledge I have accumulated throughout my studies so far (from Infrastructure, Software Development and, most importantly, Information Systems).

The second reason I am considering removing it from the project is that it is a large amount of work to add on to an already rather large project with potential of running into more considerable roadblocks while trying to get the Blockchain nodes communicating over the virtual network.

The third, and final, reason I am considering removing it from the project is that, even after working on for another full week, my platform is simply not ready to be deployed “permanently” (without modifications etc.) and this may be an issue that will be ongoing throughout my project as my research and development continues and requirements (inevitably) change.

So as it stands, I have decided to put this portion of my project on hold and focus purely on the payments and accounting systems and if nearing the end I still have some time I will reconsider building the virtual environment as I can always deploy the nodes with a copy my local “test net” Blockchain.

Seminar Session

As I mentioned last week, I had the first seminar session and Thursday. The seminar session was interesting to attend but, to my surprise and despite it being a compulsory session, there seemed to be many people who did not attend.

It was interesting to hear what other students are doing with their project and how they are progressing so far and it seems like most people are making good progress. I was interested to hear about what other students planned to do with Blockchain, and I look forward to seeing what results they have come up with in the next seminar.

Other projects I look forward to learning more about are the Ben’s AI integrated website, Benji’s Car configuration website and Will’s more tradtional web app.

Back-End Work, Signing and Paying

The back-end work this week primarily consisted of adding two functions, signInvoice and payInvoice, the first function is important from an accounting perspective because this essentially acts as the invoice authorization, and the second is important for… well… making the invoicing system useful? Some would even say that it is essential functionality.

In all seriousness though, the main function here is signInvoice as the payInvoice function was trivial to write at best and is not overly special, now customers can pay invoices, great! What’s important about the sigInvoice function is that it makes use of the Blockchains cryptography to authorize the invoice.

It works using a Public/Private key pair, where the customer uses their private key to “sign” an arbitrary message (example shown below), which is then recorded on the Blockchain as a) an encrypted string which acts as the “signature” and b) a verified transaction. This “signature” can then be retrieved from the Blockchain and decrypted. This ensures that the authorization is authentic because it can only be decrypted using the customers public key and if the keys don’t match then the signing will fail.

7 signature

Code for Example “Signature”

The above code snippet shows the example “signature” that I have come up with, this may not be accurate or contain enough information, however, when run through the function it takes a string like this:

Formal Approval for Invoice Number: 996782431 – Amount Payable: $22,632 – From: Company A – Account: bb1a3d0f52fa761fc48eb16c9fd8eb02385c8e70 – To: Company B – Account: 6044ca16e88a572900871b4f3ce791f710f6072f

And turns it into a hash like this:

0xc30a6ecc1af1fb39adeb443914cc0a4bccd4a31169048dc7bdb9ceb536b5c9cf5282b156b2f84defdb6d35e902fda4e9c5b14ca570202e4b70cd8e1fc3ad2a9c1c

Which can then be decrypted using the buyers public key for verification that the invoice was in fact approved by the buyer.

Front-End Work

In contrast to last weeks Back-End focus, this week has been all about getting information from the back end to the front end, making sense of it and displaying it in a clear format, as well as cleaning up my “testing” functions and turning them into proper functions.

Listing Invoices

After last weeks progress of being able to publish invoices to the Blockchain and get basic details back into the console, it was time to get those details into the front end.

The first step was to get the list of invoices, I assumed that this was going to be a trivial task as I was already able to get the details of each invoice individually, so I assumed I would be able to run a for loop and get all the invoices, then I would just need to format and display it, however, I had to do some additional learning about JavaScript Callbacks and Promises as well as Synchronous and Asynchronous programming.

The problem was that when the code ran, it would not wait for a response from the Blockchain before continuing, this meant that variables were left undefined while the code attempted a concurrent call to the Blockchain (which it doesn’t handle well). After some research I figured out how to get the code to wait for a response and eventually, ended up with a list like this:

1 Listing Invoices

Listing all Invoices

Success! Next, it was on to displaying all the details in my invoice template, again, this was not quite as simple as I first anticipated.

Because I was already getting all the invoice details I figured that I could simply pass the variables containing the details to a new window and use AngularJS to bind them to the various elements of the invoice template.

As with many things, this was easier said than done. The hardest part was sending the variables to the template window, however, after some further research I managed to do it using some more Angular magic ($window.open) and spent quite some time working with the bindings. The end result:

2 Invoice Template

Invoice details retrieved from the Blockchain

A fully functional, dynamic HTML invoice template. If only my descriptions were more… descriptive… 😉 Now that I could open the invoices, I needed to add the basic accounting functionality (signing and paying).

To do this, I added a small options card to the side of the invoice template which has different states depending on the invoice state, and will eventually change depending on whether the active account is the “supplier” or the “customer”, but small steps for now!

This slideshow requires JavaScript.

Goals for Next Week

Now that I have the invoicing system mostly up and running, I want to focus of using it from the different perspectives, currently it is very much in a testing type scenario where I am only using one Blockchain account to act as both the supplier and customer (to themselves) so I would like to alter this so that I can switch accounts and filter information depending on the active account.

Also, I would like to start working on the “Make a Payment” page of my app, where one account can send payments to another and include classifications for the payments.

If I manage to get all of that done, then I would like to start working on getting lists of transactions for the accounting side of the app.

According to the planned schedule in my proposal, I was aiming to start populating the Blockchain with test data next week as well. This will all be dependent on my progress and may need to be pushed back to the following week.

Advertisements

PRJ701 – 19/08/2018 – Proposal Approved!

Wow, here we are at the end of week 4 already! However, on Friday (the 17th) evening I received the fantastic news that my proposal had been approved and my supervisor will be Sandra!

This means that the last few weeks of researching and learning have put me off to a great start with my project. Now I hope to continue that momentum and keep things on track with my timeline, although, all of the tasks that I’ve set for myself have been taking far, far longer than I anticipated (as they normally do when experimenting and learning new things) so I am not as far along as I had hoped but I am hoping to pull that back next week (we should always be hopeful right?).

Back-End, Solidity Development

So my first task this week was to get started on the Back-End of my system. For this, I need to use Solidity to generate a “smart contract” that will essentially act as the “invoice”.

Thinking from a high-level, an invoice is essentially a contract between two parties, for example:

  • Party ‘A’ agrees to:
    • supply party ‘B’
    • with ‘X’ goods/services
    • for ‘Y’ price
  • and,
  • Party ‘B’ agrees to:
    • pay party ‘A’
    • ‘Y’ amount
    • for receiving ‘X’ goods/services

Now, I needed to design a way to take the information from my Front-End template and store it in the smart contract. The greatest challenge with this is that, because an invoice does not have a defined number of lines, I cannot specify a defined invoice structure, further to this, Solidity does not support relational modeling very well (because it was not designed to).

String Arrays and Solidity Limitations

In my initial attempt, I wanted to use an array of strings inside the invoice struct that would hold the line details as a delimited list, however, upon trying to put this into action I ran into another limitation of the Solidity language, which is that (at this time) it does not support passing an array of strings to, and returning them from, functions. Obviously, that ruled this option out (or so I thought, more on this later).

Relational Modeling in Solidity and Breaking ACID

My second idea was to attempt using the slightly more complicated method, found here, to create a sort of “relational” structure inside of the contract, utilizing multiple structs (one for the main invoice and one for the individual lines) and creating a sort of “primary/foreign key” relationship between them. For the start this seemed like a promising option, however, it was only after creating the contract and attempting to test it/integrate it with the front-end that I realised a fatal flaw of this design, it leaves the possibility of incomplete data being written to the blockchain and, particularly for my use case, that is not a risk that can be taken.

The reason for this is that each write to a struct has to be made as a separate transaction, one for the primary invoice details and another for each line of the invoice and once a transaction has been made it cannot be rolled back, so, for example, an invoice with two lines would require three transactions:

  • Tx1 Invoice Header Details
  • TX1 First invoice line
  • TX3 Second invoice line

and if Tx1 was successful, it is permanently on the chain, but if Tx2 fails, then Tx3 is successful, there would be an invoice that is missing a line, the record would be incomplete. This scenario is simply unacceptable for my application.

Further to this, submitting the invoice feels like an old Windows Vista security prompt nightmare because there is a prompt to confirm each individual transaction and, as far as I’m aware, it is by design, and there is no way to avoid this.

It would be an ideal scenario if I could do both the invoice details and the lines in a single transaction, but I cannot find a way to do that using this design so, because of this, I have decided to move on from this solution and look for another alternative.

Maybe a Solidity/Web3 guru might read this one day and post a comment with a potential solution for future use? always hopeful 😉

Additional Cryptography?

Another thought that I did have was finding/using some sort of two-way hash function to turn the entire invoice into an encrypted hash and simply store the hash on the Blockchain which I could then retrieve and decrypt locally to get all of the invoice details. This option would probably require a substantial amount of additional research and really exceeds the scope of this project but it was worth a thought.

Bytes Over Strings

After furthering my research and developing my knowledge, it has become apparent to me that, in general, Solidity appears to prefer the use of Byte type variables over String types, this got me thinking again about using an array of delimited information and I realised that Solidity can, in fact, handle passing an array of Byte types to and from functions so, as it turns out, I could have simply substituted the array of strings for an array of bytes and it would have worked perfectly at the start of the week!

Well… Almost perfectly, the one small issue is again due to hitting limitations within Solidity. This is because the byte “string” cannot be longer than 32 characters, which as you can see below, means some information gets lost, see on “line 3” for the freight that the quantity and discount get cut off, this should be a simple fix though.

sendingDetails

Invoice to be Sent

fetchingInvoiceDetails

Basic Invoice details in the console

Overall I am pretty happy with my progress this week. I can now, finally, successfully submit an invoice to the test network and retrieve its details, even if I can only see them in the console at this point, it’s one step closer.

Front-End Work

Although most of the work this week has been “back-end” focused, I did have some basic goals for the front-end this week too, I wanted to complete the Invoice Template and I wanted to create a menu to help me mentally scope the pages that need to be available in the prototype and their functionalities.

Menu_v1

First iteration menu for the Prototype Application

This is just a rough brainstorm of which menu options will need to be available and working in the prototype. I have split it into two categories:

  • Accounting
    • Transactions will list all transactions and their classifications
    • Balance Sheet will look at specific accounts
    • General Ledger will look at all the accounts
  • Invoicing
    • Create Invoice is where the bulk of my work has been so far, it shouldn’t need much explanation
    • Sent/Received Invoices will list all invoices either sent or received…
    • Make a payment will be for general business to business payments/expenses without invoices, maybe?
Tax_Invoice_Template

Blank Invoice Template

I didn’t want to spend too much time on the invoice template as this is just a simple prototype, this is what I slapped together, it’s not overly pretty but it is functional. When the user clicks on the invoice details button (sent/received) this is what will pop up.

Goals for Next Week

Gosh, things are moving fast! According to my initial plan, next week I need to start working on the virtualized environment and look into how I am going to deploy the multi-host multi-node blockchain network so that will be goal number one for next week.

To successfully deploy the distributed network also means that I would need the platform to be developed enough that I am able to send invoices, process their payments and also send other arbitrary payments, of which, I can currently only send invoices. So, I feel like I am slightly behind in my schedule and need to start working hard to catch up and get these elements completed as quickly as I can, so working hard on the platform will continue to be on the priority list, most notably, being able to make payments on the invoices and making any other arbitrary payments.

Also, the first seminar session will be on Thursday next week (the 23rd) which I am looking forward to as everyone will be sharing their progress and going more in-depth about their projects and I’m sure will be interesting to hear more details about what everyone else is doing.

PRJ701 – 12/08/2018 – Continued Research and Prototyping

It has been a relatively slow week this week as I am still awaiting approval for my proposal, however, I am hoping to hear back about it by the end of the coming week. In the meantime, I have been doing further research into meeting the requirements of my project and the development tools available.

Ethereum Learning

So far I have been researching “smart contract” development and looking into ways of storing the invoice details, one of the challenges is thinking about how I am going to be storing the structure of a multi-line invoice inside a smart contract, so far I have two ideas:

  1. Use an array of delimited strings inside of the invoice struct
  2. Use a “relational” method with multiple structs (more information here)

Both of which present their own unique challenges.

Front End

On top of my Ethereum learning, I have been working on designing the front end prototype, the greatest challenges here were:

  1. Making a template that allows new lines to be added dynamically
  2. Calculating the totals across the multiple lines and,
  3. Enforcing the error handling constraints

For this, I have opted to use AngularJS as it is somewhat familiar and there is a multitude of tutorials and other resources available for making dynamic HTML/JS invoices with Angular. Here is what I have come up with so far:

1 Invoice Creation

The basic idea in this template could be adapted for small/independent businesses or larger/corporate businesses/enterprises. I have designed it with the thought that the product section would be linked to a database with set products at fixed prices, which can then be adjusted with the discount field.

Currently, the design allows for multiple lines to be added and calculates the totals (line totals, overall subtotal, GST (at 15%) and grand total) in real time as variables are adjusted and error checking is done on submit (so you cannot submit an invoice with no product lines, cannot submit blank lines, must select a customer and enter a payment due date).

As the aim of my project is to create a simple prototype I won’t be diving too much further into complex features or systems, however, you could imagine in larger scenarios that the invoicing system could be integrated into a purchase order/dispatch system, both of which would be linked to some sort of product catalogue database or in smaller scenarios the user could enter product descriptions and costs manually.

But for now, I am happy with what I have. Next, I will need to design the back-end and figure out how to capture this information and I will need to figure out how to retrieve it also so that I can display it in a list, something like this:

2 sent invoices

Then I will need to figure out how to generate a complete invoice when the user clicks the “Details” button.

Currently, I am working on the invoice template and will then start working on the back end. Once that is done I’ll start work on linking it together and, finally, the accounting stuff.

PRJ701 – 05/12/2018 – Pending Approval

This week has been a busy one, pending approval of my proposal I have been spending a lot of time looking for learning resources and doing preliminary research around the different blockchain platforms that are available and how I could use them to develop my project.

Deviating from my Proposal Already

Choice of Blockchain Platform

In my proposal, I mentioned that I would like to use the EOS platform as a base to build my project on. The reason for that is because the EOS platform can process transactions much, MUCH, faster than any other platform (which is absolutely crucial for fast-paced business environments), and deploying a private blockchain which allows the exchange of currency types other than the usual (ETH or BTC) is really very easy.

Through my research and resource gathering for this platform, I managed to deploy a single-node test-net, use the CLI to set up some wallets and accounts and transfer some currency between accounts, write a basic smart-contract (however, I could not figure out how to compile and deploy it) and write a web-app that made the most basic calls to the Blockchain (transfer, list accounts).

The problem with trying to use this platform is that it is (essentially) a brand new platform (the final release version only launched in mid-June) so there are very few resources available to learn how to develop my idea on it. In fact, there are no full end-to-end resources for the EOS platform (yet) and piecing together the very few resources that are available has proven to be extremely difficult. Further to this, there are also very few development tools available for this platform and the ones that are available are rather unrefined and very time-consuming and difficult to use.

If I had more time for the project I would look to persevere with this platform as I believe that it is superior, however, because my time is limited for the project I have decided to switch to the Ethereum platform and use that to develop my project. This should be adequate as it is only a proof-of-concept project which could later be ported over to EOS to make it faster.

Prototype Application Platform

Another thing I mentioned in my proposal was that I would like to develop a prototype application to interact with the blockchain using C# or C++. This would mean I would have ended up with a desktop application that performed the basic required functions.

After a lot of research (for both the EOS and Ethereum platforms) I have found that there are far more resources (and development tools) available for creating web-based applications in comparison to desktop applications, therefore, I have decided to create a web-based prototype instead. This will mean that I am working with some basic HTML, CSS, and JavaScript, which are all languages that I have some experience with, and, as I will be using the Ethereum platform I will also be using Solidity, which will be a new language for me and Web3js, which is also new to me.

Ethereum Learning

Test Blockchain Setup

As I plan to set up a multi-host multi-node Blockchain environment for this project I will need to learn how to set up the Blockchain and launch the nodes, for this I plan on using the Geth client and possibly multiple Ubuntu Virtual Machines in VMWare.

However, we all need to start somewhere and, as this is my first time using these tools, I opted to start with a basic 2-node setup on my Windows 10 Installation. This went relatively smoothly and I ended up with two nodes running on different ports on the same machine, as well as a “bootnode” which is essentially like a router for the main blockchain nodes (it helps them connect to each other).

Geth-Test-Network

This was all made super easy to understand thanks to this guide on Hackernoon.

Learning Solidity and Web3js

Once I had my test network running I decided to dive into learning the basic requirements for developing the web application. For this, I will be using a variety of tools that, so far, all come together really, really, well. These include:

  • Visual Studio Code (As an IDE)
  • NodeJs (as a CLI package manager)
  • Truffle Framework (an Ethereum specific development tool that makes some hard things super easy)
  • and more

In playing with these I have managed to create a couple of simple apps, one that is essentially a voting system (just to get me familiar with the tools, if you’re willing to sacrifice a couple of hours check out the tutorial video I followed) and I then took the knowledge I gained from that to adapt the basic HTML dApp in this tutorial series to a Truffle/JavaScript web-app.

instructor-tutorial

Neither of these tutorials relates directly to my project, however, they were a good starting point and I should be able to use this knowledge to create the invoicing section of the web-app, from there I will need to do some more learning for the General Ledger generation (including brushing up on the required accounting knowledge) and finally migrate it all over to the multi-host network (provided that my proposal is approved of course images).

In the meantime, I’ll keep looking for resources and get started on the invoicing system, which are my current goals for next week.