PRJ701

Final Project Journal Posts

PRJ701 – 02/11/2018 – Closing Notes

This will be my final post in this journal. That means I am almost at the finish line! With the application complete my focus has been purely on writing the report but as this project comes to a close, I would like to close out this journal.

Working through this project has without a doubt been the most intense experience I have had at NMIT, in saying that, it has also been one of the most rewarding. I have really enjoyed the challenge as well as the copious amounts of learning that I have done around concepts that I have not even touched on in my other courses, all in all, I am so glad to have made it this far.

Unfortunately, I had some issues with recording the walkthrough video (the screen capture did not capture certain things and I was pressed for time while recording) and due to the current time constraints I am unable to make another recording, however, this is enough to get a general idea.

Without further ado…the final product of the project…

Advertisements

PRJ701 – 21/10/2018 – Catching Up, Completing the Balance Sheet, Devastation, Angular 6 and the Power of Blockchain

Phew! What. A. Week! This week has been all about catching up from the lack of progress lately while sorting out my new (and old) house but a lot has been done since the last post.

Completing the Balance Sheet

I had a meeting with Sandra to show my progress and she explained to me where the numbers for the Equity section of the balance sheet come from and I implemented this into the (now old) application (more on this below) so the Balance Sheet was completed and so was my proof of concept… but then…

Devastation

While prepping to record a demo run through the application, sending invoices, signing invoices, paying invoices (along with posting ledger entries) and the dashboard all suddenly stopped working correctly, after a long gruelling process of attempting to troubleshoot and research online, I discovered that one of the components powering my application (MetaMask) has released an update with a breaking change, meaning that my application was still talking to the Blockchain, however, it was no longer receiving a response and therefore the function was stuck waiting.

This was a core part of the workflow for a large portion of my application so it required a lot of work (along with more learning) to adapt to the new process especially as the code of my application was like a large can of spaghetti. Because of this, I made another choice.

Switching to Angular 6

Thus far, my application has been developed using the AngularJS (aka angular 1) framework (with Bootstrap 4 styling) and, as I didn’t spend much time learning how AngularJS works or how to use it properly my overall design was very poor and did not make use of the full potential of the framework.

Because of this and the fact that I had to re-write a large portion of the application anyway, I decided to spend some time learning how to use the Angular 6 framework and rebuild from scratch. This was not as hard as anticipated and allowed me to gain even more new knowledge by using the latest version* of another popular framework and also using the TypeScript language.

I also decided to use the Angular Material library to get some pre-built components and styling so that I could focus on the application rather than styling.

(*Note: Since writing, Angular v7 and Material v7 have been released)

New Application Screenshots

Here’s a quick run through of the new application

(Note: Updated some screenshots after a bug fix)

1-dashboard

1. New Dashboard

2-chart-of-accounts

2. Chart of Accounts

2-new-sales-invoice-1

3. Blank New Sales Invoice

3-new-sales-invoice-2

4. New Sales Invoice with Content

4-sent-invoices

5. List of Sent Invoices

5-received-invoices

6. List of Received Invoices

6-invoice-modal-1

7. Popup Preview of a New Sent Invoice

7-invoice-modal-2

8. Popup Preview of a New Received Invoice

8-invoice-modal-3

9. Popup Preview of a Received Invoice that has been Signed

9-invoice-modal-4

10. Signed invoice showing the signature

10-invoice-modal-5

11. Popup Preview of an Invoice that has been Paid

11-balance-sheet

12. Completed Balance Sheet

13-general-ledger-report

14. General Ledger Report View

14-transaction-receipt-1

15. New Transaction Receipt Feature (Invoice Deployment Receipt)

15-transaction-receipt-2

16. New Transaction Receipt Feature (Invoice Payment Receipt)

16-transaction-receipt-3

17. New Transaction Receipt Feature (Invoice Approval Receipt)

17-pnl

18. Profit and Loss Report

Transaction Receipt Feature

As I have mentioned in a previous post, my application does not fully meet my original expectations when it comes to automating the bookkeeping process with Blockchain alone. This application does allow for bookkeeping and provides some basic functionality to demonstrate this, however, it is still being achieved via a centralised database and manually selecting when and where debits and credits are applied.

The centralisation of a companies personal accounting records is not a huge concern, and companies will often entrust the storage of their records to a third party (i.e. Xero or QuickBooks etc.) and the actual upkeep of those records to another third party (i.e. their Accountant) and this model is actually quite reasonable.

I have also stated before (and still feel) that true/full automation of bookkeeping is more of a job for an AI system, not Blockchain, however, I do believe Blockchain, along with triple entry accounting, does offer the potential for something truly unique and extremely powerful for the accounting space.

This has been a very long introduction to a relatively small and simple new feature that has made its way into this version of my application, the Transaction Receipt (Seen in Figures 15, 16 and 17 above) but I feel that this feature deserves this lengthy introduction.

Anyone that has used cryptocurrency before could think, well, so what? I can just jump on etherscan or blockchain.com and see the details of any transaction, what’s so special about that?

Quite simply, it is the context that these receipts give to the transactions, by reviewing and verifying some relatively simple data points, it is not only possible to verify that a transaction took place but actually see the reason for that transaction.

Again, this somewhat defeats the purpose of traditional cryptocurrencies that are supposed to provide a means for making anonymous transactions without disclosing the context or “reason” for the transaction but, from an accounting and business perspective or, more to the point, an auditing and government perspective, being able to verify that transactions took place on the fly would be extremely powerful.

Similarly, being able to use this information at a later date to ensure that each transaction has been recorded accurately in the accounting records of the involved parties would mean that the auditing process could, one day, become somewhat, or even completely, automated. And that is the real power of Blockchain in Accounting.

Goals for the Coming Weeks

From here, I will be mostly focused on writing my final report for this project. I also have the last seminar session coming up and will continue to add a few transactions each day. I will also re-attempt a video walkthrough of the final application.

PRJ701 -07/10/2018 – Profit and Loss, Balance Sheet (sort of) and the Written Project Report

This week has again been slow in terms of development. I have implemented the Profit and Loss report and (almost finished) the Balance Sheet, however, we received some pretty abrupt notice that we need to move house so things have been crazy while I have been helping prep for our move next week so there may be very little to add next week as well.

Profit and Loss Report

pnl

Example Profit and Loss Report

The profit and loss report shows the values that make up the Gross Profit and the Operating Expenses, the Net Profit is then calculated as the Gross Profit less the Operating Expenses. This value can then also be used in the Balance Sheet

Balance Sheet

balanceSheet

Example Balance Sheet (incomplete)

The balance sheet takes the Accounting equation of:

Assets = Liabilities + Owners’ Equity

and changes it slightly to:

Assets – Liabilities = Owners’ Equity

Which in the end balances just the same. The issue I am having currently, however, is that I am getting confused with how to calculate the remaining portion of the Equity section in the Balance Sheet. Depending on where I decide to go with the rest of the project I will get in touch with Sandra to organise a meeting and discuss this once I have settled into my new house.

Starting the Report

Looking at the remaining timeline for the semester is getting pretty scary now, with the Journal due in just under 4 weeks and the Report due the following week, not to mention the Poster being due the week after!

In light of this, I am considering my options with my remaining time. Originally, I had planned on having completed the work on developing my app and adding test data by the end of the “break” (from classes), leaving the remaining time to work on my Report and Poster.

Currently, my application does meet the initial brief in terms of making triple bookkeeping entries and providing the absolute basics of bookkeeping (some of the extra reports are not fully complete, however, they were not part of the original brief anyway). What is missing is the test data. I did want to include at least one full month of test data, and this may still be possible, but it would put me very close to the due date of the Report and I don’t know if I want to do that.

With my app semi-complete and not really proving the concept that I set out to prove, I am feeling a bit lost with starting my report as things have changed quite a lot since writing my proposal.

I also feel a bit lost amongst the excitement, stress, and general chaos and confusion associated with moving house and I feel that once we have settled in the new house I will be able to put a more solid focus on my project again.

However, I have started drafting out the report sections and layout, along with adding a few paragraphs and notes here and there.

Goals for Next Week

I will hopefully move house successfully next week and that will be a big goal, although not related to the project directly, getting that done will help me focus more. I will also aim to start writing more seriously on the report while I ponder what to do regarding the test data and any further development.

PRJ701 -30/09/2018 – Switching to a “Multi-Contract” Design, Chart of Accounts and the General Ledger

Compared to last weeks effort, this week has felt like a major step forward. My first port of call was to switch from using a single smart contract as a “controller” to a using a base contract as a “template” and deploying a new contract for each invoice. After I implemented that into my code I got moving on the accounting side of the system.

Multi-Contract Design

Switching from my original design to the new design turned out to be very simple to accomplish and, overall, the new design simplified many other aspects of my application, along with the help of capturing data with MongoDB.

InvoiceFlow

Simplified Diagram of Data Flow Between Designs

The above diagram shows a simplified representation of how the two designs work and why they are very different. Although the “Template” diagram looks more complex, it actually simplified my code significantly and is far more elegant overall.

“Controller” Design

With the original “Controller” design, the single document would get larger and larger as each new invoice was created. This meant that the performance also decreased as new invoices were added because, to find one record, I would have to pull the entire document out, then look over the entire document and filter each piece of data that I needed, one by one.

“Template” Design

With the new “Template” design, I can grab one invoice at a time and look specifically at the information I want and, thanks to the additional data stored in MongoDB, I can filter the invoices I grab by a specific value, like by date or recipient to name a couple of examples.

Chart of Accounts

a COA-all

View of Chart of Accounts

After getting the ledger accounts set up in MongoDB and capturing the necessary data, I needed a way to display that information in the front end. This was surprisingly easy using a combination of NodeJs/Express with the Mongoose API and Angular. It’s not pretty but this is only a proof of concept.

General Ledger

Finally, what the project set out to accomplish, creating a General Ledger. Although the path here has changed the projects main idea drastically, this is what I set out to do in my proposal (kind of) from my proposal:

“This project aims to utilize Blockchain Technology and simulate a business to business transactional environment based on a shared distributed database and develop a prototype system or application to analyse the database and extract the required information to automatically generate a general ledger for each simulated business.”

The keywords that have changed in bold, the system I have come up with is only partially based on a distributed database and the General Ledger is actually being generated with the information in a centralised database. Oh well, I have done a great deal of learning on the journey here, especially around full-stack development (which I have enjoyed far far more than I thought I would) and certain aspects of accounting so I am happy with this result.

I decided to display the General Ledger in two “views”, the first being a summary view which simply lists all the accounts with their month to date (MTD) figures and, of course, that all the accounts balance.

b GL-summary

General Ledger – Summary View

The second view is a more detailed view which shows when and where the transactions occurred within each account. The major additional information in this view is displayed under the reference column, where the Blockchain transaction hash is displayed as proof that each transaction definitely happened.

c GL-report

General Ledger – Report View (Top)….

d GL-report2

General Ledger – Report View (End)

Once again, this shows that the accounts balance. The only thing missing from the report is the “Type” column which I am unsure if I need to include, however, I think it would be nice for completeness.

Goals for Next Week

I have started with a very small amount of test data to get the figures that are shown above so next week I will start to add some more invoices to increase the available data which will be perfect as Monday is the start of October.

On top of this, I may work on some other reports that have been suggested by Sandra, one being a Balance Sheet and the other a Profit and Loss report. But from here, I will work on my final report as I continue to add data.

I will also look to make a nicer demonstration of the working application most likely in video format (as also suggested by Sandra).

PRJ701 – 23/09/2018 – Posting to the Ledger Accounts

Work on my project this week was rather slow, with the due date of my DAT701 assignment looming I had to put in more work for that than I was initially expecting to, although, I did manage to settle on a design for the ledger account schema in MongoDB and get started on the API.

This Weeks Achievements

I mentioned last week in my “goals for the coming week” that I needed to get the inventory system and accounting system up and running, capturing the required data and that is about all that I have achieved this week so this will be a relatively short post.

But, I can now track a sale through my system from the creation of the invoice, to approval of the invoice and final payment. These changes all reflect in the ledger accounts and I can view them in the MongoDB GUI as shown below

accs1accs2

accs3

There was a minor issue in my code that meant the “txHash” for the debit to Wicker Basket Inventory was showing the block number instead, however, this is now fixed.

accs4

Changes are also recorded in the Invoice and Inventory collections.

invinventory

There are still some details that need ironing out, such as the “reference” numbers on the transaction objects, these do not match currently as half of the code is using the invoice number from MongoDB and the other half is using the number from the smart contract on the Blockchain.

I am still deciding how I will manage this, however, now that I have bitten the bullet and integrated another database, the approach that I will most likely take is to change my current “single smart contract” approach to managing the invoices and deploy a new smart contract for each invoice.

Doing this shouldn’t be too hard to implement into my current design and should allow me to speed things up with testing/inserting data (currently it takes almost 1 minute to load the sent/receive invoice pages) and I believe that it is a more robust design in general.

Goals for Next Week

Next week I will look briefly into the idea of deploying a new contract for each invoice, if this does prove to be simple to implement, I will do this first, if not, I will work with what I have and align the reference numbers in the transaction details, then finally, FINALLY start adding some proper test data by the end of the week.

PRJ701 – 16/09/2018 – Adding MEN to my Stack, Perpetual Inventory and More Accounting

Rationale: “The Ethereum Blockchain just isn’t cutting it for storing the Inventory data and accounting records… I need another form of non-volatile storage to hold all the inventory data and accounting records, one that’s fast, supports ACID and doesn’t have a confirmation popup for every insert or update in a transaction.”

Impulse: “Hey, I’ve got a great idea!”

Rationale: “Oh really? What is it??”

Impulse: “Let’s try to teach ourselves how to use MongoDB in an afternoon and use that to store all the inventory data and accounting records!”

Rationale: “Hmmm… That doesn’t sound like the best idea… trying to implement MongoDB could be more complicated than just learning how to u…”

Wreckless Decision Making: “LET’S DO IT!!!”

Problems With the Solo Blockchain Back-End

I have mentioned in a previous posting that I wanted to try and eliminate the need for a “traditional” centralised back-end database in my app, however, after attempting to implement the accounting functions into the app using only the blockchain, I ran into a few issues.

I believe most of these issues are a limitation of the Ethereum platform for my application, however, they were enough for me to reconsider my options when it came to achieving more “completeness” with the functions that I want to implement in the prototype.

Making “Inserts”

Ultimately, the big issues come from the speed (or lack of) of the Ethereum blockchain, the lack of ACID “transactions” and the gas system.

To implement the perpetual inventory system in conjunction with trying to create and maintain the accounting records I need to do multiple additional operations for:

  1. Creating an Invoice
  2. Signing an Invoice
  3. Paying an Invoice
  4. Receiving Payment for an Invoice

Essentially for the first two functions, I need to update the inventory numbers (current SOH and total current value) and for all three I need to add the appropriate debit and credit entries to the appropriate ledger accounts.

The obvious choice for consistency in this situation would be to bundle all of these operations together in a single “transaction” for each function which for the last 3 functions would actually be okay. The issue arises when creating an invoice which requires the following actions:

  1. Post the Invoice to the Blockchain
  2. Calculate the Average cost of each inventory item
  3. Transfer the cost of each item and the sale price to the ledger accounts
  4. Update the inventory

Sounds simple enough, however, when we break up the logic of this “transaction”, it looks more like this:

  • Update Invoice Struct with New Invoice Details
  • For each invoice line
    • Convert bytes32 into a string
    • split string by the delimiter into an array of SKUs
  • For each SKU in SKU array:
    • Get the current inventory values (SOH and Total value) from Inventory Struct
    • Calculate average cost and store in a local variable
  • Update Accounts Receivable Account – debit the invoice grand total
  • For each SKU in SKU array:
    • Guessing that it gets to about here before:
    • ———-EXCEEDED BLOCK GAS LIMIT ERROR———-
    • Update SKUs Sales Account – credit line total
    • Update SKUs COGS Account – credit SKUs average cost
    • Update SKUs Inventory Account – credit SKUs average cost
  •  For each SKU in SKU array
    • Update Inventory Struct – deduct qty and (average cost * qty)

And here we can start to see the problem, to do all of this in one “transaction” and keep it atomic requires that all of the computation is done on the Blockchain side, which, as it turns out is quite a heavy burden for the Ethereum blockchain, too much in fact as I kept getting the error shown above in red.

Splitting this function up into smaller parts would allow it to “work” however, this implementation would mean the transactions are not atomic and could result in inconsistencies with the records and when it comes to accounting, that is simply not tolerable, not to mention the long wait times as each portion needs to be completed in sequence and the MetaMask popups for each one, over 5 of them actually…

Performing “Selects”

Another issue I have been having is to do with retrieving (“selecting”) data from the blockchain, it is cumbersome at best and a nightmare any other time.

Despite refactoring as best I could, just getting a list of all the invoices and turning it into “human-friendly” information takes 5 different functions, that doesn’t include actually displaying them on screen or allow for any kind of filtering (i.e. dates, companies, status) every single invoice needs to be pulled from the blockchain into memory and processed locally each time, with less than 100 “test” invoices it takes about 40 seconds, now imagine doing that for a corporation with hundreds of thousands of invoices.

Then imagine a company with thousands of inventory items and ledger accounts… pulling ALL of that information down to perform operations on just a few would be ridiculous.

And alas, this is where I have had to make an important decision and I have come to a major finding in trying to build my project.

Implementing the “Extended” MEAN Stack

1 Traditional MEAN stack

Traditional MEAN Stack

In order to combat the issues with the Blockchain, I decided to implement the rest of the MEAN stack (as I am already using Angular.js for the front-end) which as shown above includes: MongoDB, the Express framework, Angular.js and Node.js.

As I will also be using Ethereum as a back-end, I have coined my current set up the MEEAN stack, shown below.

2 Modified MEEAN stack

“Extended” MEEAN Stack

This allows me (and potentially many other developers that use this (or the MERN stack which uses the React framework instead of Angular)) to do many things that are otherwise not possible while using Ethereum as the sole “back-end” or database.

The main function that the MongoDB serves is to provide a lightning fast non-volatile storage solution with each entry containing a “pointer” to the information stored on the Blockchain. So, for example, the Inventory Ledger accounts records for an item would look something like this (i’m yet to finalise the design) in the MongoDB database:

{
accountCode: 601
name: [SKU/Item name] Inventory
category: Asset
debits:
  {
    {
      date: 12/09/2018,
      amount: 200,
      blockchainTransactionHash: 0xabcd1234567890
    }
    {
      date: 15/09/2018,
      amount: 200,
      blockchainTransactionHash: 0x2468abcdefghij
    }  
  }
credits:
  {
    {
      date: 13/09/2018,
      amount: 200,
      blockchainTransactionHash: 0xlmno0987654321
    }
    {
      date: 16/09/2018,
      amount: 200,
      blockchainTransactionHash: 0xpqrs2468135790
    }  
  }
}

This would allow me to do a few things, firstly, I can query the MongoDB and return only the records within a specific date range (i.e. the current or previous reporting periods) and secondly I can use the “blockchainTransactionHash” value to validate the records.

Perpetual Inventory System

I will also be using the MongoDB for the inventory system, currently, I have implemented the “sales” side of the inventory system which means when an invoice is created the inventory items document is updated with the new numbers (SOH and Total Value) and all of an items transactions are stored as a sub-document, it looks like this in the database:

I am yet to finalise where the invoice numbers will be coming from in the system (currently it is using only one incrementing number on the blockchain which means that companies cannot have their own numbering schemes) so the “Ref” field doesn’t contain the invoice number yet but this provides an audit trail for where the inventory is coming from or going to.

What Else Can it Do?

Another example of where this could be used (although I won’t be implementing it), it to monitor the blockchain and check each new block as it is added to determine whether it contains any transactions, and make a records of all the blocks that do alter a balance with the date and block number, this would allow me to see the “historical balance” of a blockchain account for a particular period (something that is currently not possible with Ethereum alone).

Considerations

There are, however, still a few things to consider with this stack design particularly for an accounting application. The main one is that, yes, it is still possible for there to be an inconsistency between the records on the Blockchain and in the MongoDB, the risk of this could potentially be reduced by using a messaging service like RabbitMQ but that definitely falls into the “work that could be done in the future” basket.

I hate to keep going back to this, however, EOS has a MongoDB plugin built-in which would actually do all of this for me but I have come too far with the Ethereum development to switch over now and still there are no great tutorials on building a front end for an EOS dApp.

Is MongoDB Really Viable for an Accounting Application in 2018?

Anyone reading this that has developed an accounting system might be thinking, but Corben, MongoDB is an awful choice for an accounting database! And, up until version 4.0 that may have been true.

One of the biggest arguments has been that MongoDB didn’t have ACID support for multi-document updates, which is inherently a trait that is required by an accounting system, however, this all changed on the release of MongoDB version 4.0 when they introduced proper multi-document “transactions” with full ACID support, meaning that, just like a relational database, all records are updated or everything is rolled back.

Another argument is that relational models are the perfect design for storing accounting data because, unlike most real-world data, accounting data is actually tabular and fits extremely well into a relational model. And actually, I have to agree with this argument, a traditional relational database would most likely be a better fit for my application, however, I believe that MongoDB will handle it just fine and I went with this option for two main reasons:

First and foremost, because over the last few years I have had a lot of interaction with SQL databases including MS Access, MySQL and SQL Server, but thus far I have never used MongoDB or any NoSQL databases for that matter, so I’m doing it for the experience (another plus is that this also gives me experience using NodeJS and Express).

And secondly, because it just seemed much easier to implement into my current stack, no need for creating a normalised design, working out joins or Object Relational Mapping.

This late in the game trying to add the kind of complexity that would come with an SQL design is a bad idea, plus my experience in using the various SQL systems as well as various programming languages translated rather easily to MongoDB as the concepts are similar but, in my opinion, far simpler, which meant I really could pick up the necessities in an afternoon, learning to use Express and the MongoDB API (and Mongoose) to build a RESTful API was actually much harder, but a lot of fun.

Goals for Next Week

Phew! This brings me to the conclusion of another week and to making plans for next week, this week was quite heavy although I have not moved forward much it has involved a lot of problem-solving, causing me to neglect my DAT701 assignment quite a lot (Sorry Nick! and myself 😓) but next week, unrelated to the Project: I need to finish my DAT assignment.

Related to the project, I need to really get cracking on the back-end of the accounting system and finish up the inventory system so I can finally get some data in! Only 3 weeks behind schedule…

this-is-fine.0

But the delay has come because I now realise that the back-end of the accounting system needs to be functional (i.e. capturing the necessary data in the ledger accounts and inventory) at the time invoices are being created, signed and paid so I cannot begin to add test data until this is functional so that will be the only goal for next week.

PRJ701 – 09/09/2018 – Accounting, Accounting and more Accounting

I stated last week that I wanted to increase the number of Blockchain accounts and products that I have in my system and to finally start generating test data, however, this week I have spent a great deal of time refreshing and trying to extend my Accounting knowledge so that I can analyse/organise the transactions into debits and credits against the appropriate accounts, I had a meeting with Sandra to discuss some details and she provided me with some learning material to work through.

The reason that I did not start generating the test data this week is that I feel like I may need to modify my current system further to include additional information about each transaction in order to get the accounting side of thing correct.

Initial Gathering Transaction Information

At the beginning of the week, I started working on gathering and organising the information that relates to the different invoices, this resulted in me plotting the information on a couple of graphs

accountingDash

First Display of Accounting Dashboard

The “Accounting Dashboard” is a new section of my web application that will most likely “tie-in” with the other Accounting sections (Balance Sheet and General Ledger) as a single page app because they will all require fetching the same invoice data from the blockchain to generate.

Meeting with Sandra

Chart of Accounts and Inventory Systems

In my meeting with Sandra, we went over the chart of accounts that I added to last weeks post, and we discussed the logistics of accounting around inventory and the “triple entry” example I posted.

In the example (re-posted below) I provided, it assumes a perpetual system for accounting for inventory, which, in laymen’s terms, means that each inventory item is accounted for at the time of a sales entry.

triple entry example

The advantage of this is that accounting for inventory is done in “real time” meaning that, at any given time a business will know the value of their current available inventory. However, the trade-off is that accounting with this kind of system is logistically more complex.

At the time Sandra suggested that I should instead use a periodic system in order to simplify the accounting. As the name suggests, in a periodic system inventory is accounted for (reconciled) in periods rather than item by item, which would remove the need to account for the inventory at the time of the “sale”.

The disadvantage of this is that the inventory is not accounted for in “real time” which takes away slightly from the original idea of completely real-time automated accounting, however, shifting from a periodic system to a perpetual system could be left as a “pending” or “incomplete” use-case for the prototype.

In saying that, Sandra later sent me an email explaining how I could actually use a perpetual system instead, I have only skim read it so far so next week I will read it properly and look at how I can implement this.

The “Ultimate”(…or just better) Blockchain Idea for Business Use

It is stated in my proposal that my research is based on an “idealistic and theorised” scenario where businesses would utilise a shared platform for mutual benefits. However, I have not really provided a great context for this system, but this is something that I discussed with Sandra so I thought that I should share my ideas.

infoGraphicChains

How the System Should Work

The above infographic outlines a basic representation of how the system would work in terms of transferring “value” on a national or international basis, solving what I call the “volatility problem” of current cryptocurrencies such as Ether or Bitcoin.

In my view, the biggest problem with current cryptocurrencies is that their “value” fluctuates wildly, which means that making purchases in these currencies poses a huge risk of either paying far too much or, less likely, far too little.

This happens because 1 Bitcoin/Ether will not be worth the same amount today as it is tomorrow, or sometimes even in a few hours, this means that if you purchase something with $100NZD worth of bitcoin today, that same amount of bitcoin could be worth $200NZD tomorrow, which essentially means that you paid twice the intended price. (see this video where Olaf Carlson-Wee explains how he “accidentally” made some of his friends “thousands and thousands of dollars” just for buying him pizzas in exchange for what was then only $20USD worth of bitcoin).

The difference with my design compared to the current volatile cryptocurrencies is that, in this design, 1 local “token” would always be worth 1 denomination of the local currency, nothing more and nothing less.

For this to work, every country would need its own local “side chain” for exchanging its local token and a global chain would be needed to manage international transfers making sure they are made at the current global exchange rates. There would also need to be a system in place where tokens could be exchanged for fiat currencies.

This would work well for business to business transactions, however, when it comes to general consumers dealing with a business (which far exceeds the scope of my project but was worth a thought), it could be imagined that they too would have access to the local chain and use it in a similar way as they currently do with their banks, either with an account to account transfer or possibly even an “EFTPOS” like transaction using either specialised cards or, more likely, NFC payments from a digital wallet on their smartphones.

Logistics of Purchasing

poFlow

Purchase Order Cycle

This is just a basic flow diagram of purchase orders, showing Company A sending a PO to Company B. The idea is nothing new and it is something that also greatly exceeds the scope of my project but I feel that it is worth a thought for additional context. Thinking that this entire flow could also be tracked on a blockchain as a part of the overall system could mean that inventory could potentially be easily serialized, tracking products from the original order right through to the sale.

In “theory” this information could be used to remove the need to either calculate a weighted average cost or using the Fist in First out (FIFO) method for the Cost of Goods Sold accounting entries, however, this is something that I really don’t know enough about to speculate so I could be completely wrong with this.

Accounting, Accounting, Accounting

This is where the bulk of my work has been this week. I have been focussing on how I will integrate the accounting side of the system and what information I need to gather, what information I may need to have fixed “demo” data for to make the general ledger/balance sheet “complete” and how I can automate the process as much as possible.

accountingCert

Lynda.com Bookkeeping Certificate

I completed a basic Bookkeeping course on Lynda.com to refresh and extend my knowledge, which gave me a shiny Lynda certificate, so yeah… I’m practically a fully Certified Accountant now… right? 😎🤣

On a more serious note, the original idea of my project was to make the whole bookkeeping process automated and immutable, however, thus far I have simply made an extremely basic invoicing system with integrated payments that do have an immutable audit trail but this system couldn’t realistically select the appropriate accounts for an invoice. As my learning progresses in the accounting side of the system, I am coming to realise that I may not be able to completely automate the bookkeeping process in a way that would scale for generic use by any kind of business.

This is not to say that I think that it would be impossible, however, I just now believe that it would also require the assistance of some form of AI/Machine Learning, which is another step way, WAY outside the scope of my project, however, platforms like botkeeper and SMACC come to mind.

Nonetheless, I will focus on the main objective of my project and aim to deliver a triple entry system as a point of discussion around how businesses *could* use this kind of system to potentially automate their bookkeeping.

What I need to remember for my project thus far seems fairly basic and comes down to four main things, three are shown below, with the 1) basic accounting equation, 2) the effect that debits and credits have on the different types of accounts, 3) what kind of balance each account type has and the 4th thing, not shown below, is knowing which accounts to use for the various functions of my system.

debits and credits increase or decrease

Reference Material (Accounting Equation, When to Debit or Credit and Typical Balance Type)

Goals for Next Week

It feels like my progress has slowed down considerably this week with only minor achievements, however, I feel like my knowledge is improving and that will help me in the long run.

I do feel as though I am behind schedule slightly and I will need to focus to stay on track, however, I now also have my DAT701 assignment to complete so that will no doubt take away from my work on the project for the next couple of weeks.

Next week my biggest objective is to jump back into coding so that I can attempt to get the information from the invoices turned into debits and credits on various accounts and toy with what changes I need to make to get things moving.

PRJ701 – 01/09-2018 – Transactions, Refactoring, Accounting and the Xero Demo Company

This week has been another busy one, most of my time has been spent on learning and refactoring while I consider my options to overcome the current issues I am facing. Because tomorrow is Father’s Day in New Zealand and, being that I am a proud father (with a fantastic father of my own), I look forward to spending that day with my family, so I am shaking things up and posting this a day earlier than I have been. So let’s jump into it!

Arbitrary Transactions

One of the goals that I set for myself last week was to complete the “Make a Payment” page of the application and to work on the “transaction list”. The first part of this goal was actually surprisingly easy in comparison to everything that I have done so far! I made a simple UI, and wrote some, again, surprisingly simple code, which allowed my application to send Ether from one account to another with a custom “data” field.

make a payment ui

Basic Make Payment UI

This is the basic UI that I came up with, it allows the user to select an account from their “list of accounts”, enter an optional reference number for the payment, select the accounting categories and write some notes about the transaction. Once the user clicks “Submit” this information gets “packaged” up using this code:

transaction code

Packaging the Transaction

First, the code runs off to another function to make sure the users’ input is valid, once this check is done the transaction is put into a variable, that includes the basic to/from and value but it also includes a “gas” value, which is specific to Ethereum and is a necessary evil, but the magic happens in the “data” field. Here, the reference number, the categories and the notes get turned into a reversible hash which will eventually be stored on the blockchain as part of the transaction. Once this is done the transaction variable is sent to this function:

transaction code 2

Sending the Transaction

Which actually sends the transaction to the Blockchain. This was all well and good, and super simple! The problem, however, arose when I looked at trying to get the transaction details back from the Blockchain, as it turns out, when it comes to arbitrary transactions like this, there is no simple way of retrieving all the transaction related to a specific account (on the Ethereum platform… +1 for the EOS platform 😮 ) This is something I spent hours on and actually managed to crash my laptop a couple of times trying to accomplish.

Because these transactions are not “indexed” you cannot simply call a function that returns them, this means that in order to get all of the transactions related to a specific account, you need to run all of the blocks, right from the genesis block all the way to the current (“latest”/”pending”) block, through a series of “if” statements.

What’s the big problem with that? Well, it is extremely slow and it seems to require a LOT of computing power to scan more than a couple of thousand blocks. When I attempted to scan my entire local blockchain (consisting of just over 150,000 blocks) Chrome ended up eating 100% of both my RAM and my Processor, grinding my computer to a halt and eventually crashing Chrome along with the test web server as well.

The good news, however, is that if you already know the transaction hash, you can get all the transactions details really quickly. But how do you ensure that the hash is known? Well, there would be a few options here. Ethereum does store and indexed log of information when an “event” is emitted on a contract, this means that:

  1. We can use a Smart Contract to handle the payment OR,
  2. We can send a Smart Contract the Hash after the transfer confirmation

Another option would be to store the hash in some other form of database at the time of the transaction, however, since I am aiming to remove the need to do this, and this is a limitation of the Ethereum platform which could be solved in the future or with a different platform, I first opted for option 2 above.

You may notice that inside the snippet above of the “submitPayment” function that there is another function call circled in red, this essentially makes another call to a new smart contract that I created which stores the transaction hash. One problem with this is that it requires another confirmation from the MetaMask plugin and if the user was to click “Cancel”, or even if the contract call failed, then the transaction hash would be lost.

As this is a limitation of Ethereum I chalked this clunkyness up to the application being a prototype and this allowed me to produce a list like this:

transaction list ui

List of Transactions with Categories, References and Notes.

But this is where I decided to stop for the moment so that I could take some time to do some more research and evaluate where I was going (which I will discuss more below in the “Accounting” section).

Refactoring

Before I got stuck into my research I took a long hard look at my code thus far and thought to myself… WHAT AN ABSOLUTE MESS!!!

  • Messy/unformatted code
  • Random console.logs everywhere
  • Unnecessary variables
  • Repeated code
  • Code in places it shouldn’t be

My application had it all and all sorts of other over complications so, before proceeding to make it more complex, I decided I needed to spend some time simplifying what I had, making it more efficient and easier to read.

This took a considerable amount of time, however, I felt it was well worth it in the end and will make my life easier moving forward.

Refactoring doesn’t usually lead to adding new features, however, a natural side effect of my refactoring process also helped me to partially achieve one of my other goals from last week, and that was enabling the use of multiple accounts in the application so I could really start sending “invoices” to different accounts and see this reflected in the list of invoices.

The Xero Demo Company

So, before moving forward with further development, I wanted to do some more research into the Accounting requirements before I started developing things that were potentially an unnecessary waste of time.

What better way to do that than to evaluate a real, well polished, application that does many of the things that I’m aiming to achieve in my project. Enter Xero, you know, only one of the best accounting applications around at the moment. Xero is kind enough to have an offer available to students which allows them to start a free trial to use and explore their applications features with a demo company (this is also available to everyday users as well) which is something that Sandra mentioned to me in our first meeting.

Looking at the overall layout and features of Xero really opened my eyes as to where I needed to be going and, to my surprise, I already had a lot of the necessary information bouncing around my application to produce some of the reports, I just need to organise it! (more in “Accounting” below)

Seeing the way that Xero can directly integrate a business’s bank account also got me thinking of ways that Xero could integrate Blockchain into their system, other than with Veem (insert shameless plug in case someone from Xero happens to read this and wants to hire me 😉 ).

Accounting

After reviewing what I have completed so far and looking at a real accounting solution, the logistics of my system have started to organise themselves far better in my head and I feel that, from here on, I will be referring to Xero’s features and layout more often to keep me on track.

More on the “make a payment” page: Since looking at the Xero application I have decided that I will remove this from the scope of my prototype, this is because I feel like I can achieve my project goals adequately without this feature and, for the purpose of my prototype, things like utility payments etc. can be achieved with a bill (aka an invoice) if necessary.

More on organising the information: In order to organise the information that I have, I need to create a basic Chart of Accounts that debits and credits will be going in and out of, this is what I have come up with so far:

Chart of accounts proto

First Iteration Chart of Accounts

I would like to check this off with Sandra and see if she thinks that it would be suitable for the system I am creating or, even better, if it could be simplified.

Another idea I need to wrap my head around is how the triple entry would “look” to put it in a visual context, take the example below, which shows a simplified version of how the triple entry would work if Company B was to purchase an inventory item from Company A on account (aka using my applications invoice system).

triple entry example

Triple Entry Example for Purchase of Inventory

My next challenge will be collecting and organising the information from the current system so that I can display it in a way that makes sense in terms of the “double entry system” using the support of the triple entries, depending on the account that is currently logged in with the MetaMask extension.

Finishing Sent and Received Invoices

After refactoring my code and enabling the ability to send invoices between different accounts instead of just using the single test account, it was time to finish up the front end. Now when a supplier opens one of their sent invoices they can see the status at the top, and the header colour changes depending on the supplier (not overly fancy but small steps and… it’s only a prototype 😀 )

And, as I included in last weeks post, the receiver of the invoice can sign (approve) and pay the invoices with the floating options on the side.

Goals for Next Week

Next week I would like to expand the number of accounts that I can “interact” with within the system. The end goal of the project is to have a small business transactional environment so I will be primarily focused on populating just one or two of the “companies” with test data while the other “companies” will be there to provide “diversity” and certain types of bills/invoices. I would also like to change/expand the current product list to something larger and more realistic.

Both of these goals should be much easier to achieve after refactoring as I now have company and product details stored in JSON files instead of being dotted all over the place which makes adding companies or products as simple as adding another block in the JSON file.

Once I have all of that in place, I will reset my local Blockchain and, finally, start populating it with the test data, then I can get more seriously started with working on the accounting side of the application.

PRJ701 -26/08/2018 – [insert great blog title here]

Another week down and another round of progress towards the end goals. This week has been eventful, I had my first meeting with Sandra and participated in the first seminar session, as well as made some more progress in terms of the development of the project artifact and contemplating a decision around the virtual environment portion of my project.

Meeting with Sandra

As mentioned above, I had my first meeting with Sandra, in this meeting we discussed my overall project and my main objectives for the project, we also discussed the potential implications of my project being taking from a prototype into production on a wide scale, which was an exciting prospect for us both.

We also briefed over the potential change I am considering (more on this below) and I demonstrated what I had achieved so far.

After my meeting with Sandra I feel even more excited to work on this project and feel extremely confident that she will be a fantastic supervisor overall. Further to this, having the additional support of her advanced Accounting knowledge to support my fairly basic and limited existing knowledge fills me with even more confidence.

Another Potential Change to my Original Proposal

I stated in my previous blog that my main goal and focus for this week would be to set up the virtual environment and deploying the Blockchain across the multiple hosts.

My initial plan was to run multiple instances of Ubuntu within different subnets and connect them with one (or more) instance(s) of VyOS, however, after my discussion with Sandra I have started contemplating a change in my plan to remove the virtual environment component from the overall scope of my project.

The primary reason I am considering dropping this component of my project is because I am unsure if it will really add any great benefit towards the actual end goal of my project, my reason for including this component as part of my project in the first place was purely so that my project made strong use of all the knowledge I have accumulated throughout my studies so far (from Infrastructure, Software Development and, most importantly, Information Systems).

The second reason I am considering removing it from the project is that it is a large amount of work to add on to an already rather large project with potential of running into more considerable roadblocks while trying to get the Blockchain nodes communicating over the virtual network.

The third, and final, reason I am considering removing it from the project is that, even after working on for another full week, my platform is simply not ready to be deployed “permanently” (without modifications etc.) and this may be an issue that will be ongoing throughout my project as my research and development continues and requirements (inevitably) change.

So as it stands, I have decided to put this portion of my project on hold and focus purely on the payments and accounting systems and if nearing the end I still have some time I will reconsider building the virtual environment as I can always deploy the nodes with a copy my local “test net” Blockchain.

Seminar Session

As I mentioned last week, I had the first seminar session and Thursday. The seminar session was interesting to attend but, to my surprise and despite it being a compulsory session, there seemed to be many people who did not attend.

It was interesting to hear what other students are doing with their project and how they are progressing so far and it seems like most people are making good progress. I was interested to hear about what other students planned to do with Blockchain, and I look forward to seeing what results they have come up with in the next seminar.

Other projects I look forward to learning more about are the Ben’s AI integrated website, Benji’s Car configuration website and Will’s more tradtional web app.

Back-End Work, Signing and Paying

The back-end work this week primarily consisted of adding two functions, signInvoice and payInvoice, the first function is important from an accounting perspective because this essentially acts as the invoice authorization, and the second is important for… well… making the invoicing system useful? Some would even say that it is essential functionality.

In all seriousness though, the main function here is signInvoice as the payInvoice function was trivial to write at best and is not overly special, now customers can pay invoices, great! What’s important about the sigInvoice function is that it makes use of the Blockchains cryptography to authorize the invoice.

It works using a Public/Private key pair, where the customer uses their private key to “sign” an arbitrary message (example shown below), which is then recorded on the Blockchain as a) an encrypted string which acts as the “signature” and b) a verified transaction. This “signature” can then be retrieved from the Blockchain and decrypted. This ensures that the authorization is authentic because it can only be decrypted using the customers public key and if the keys don’t match then the signing will fail.

7 signature

Code for Example “Signature”

The above code snippet shows the example “signature” that I have come up with, this may not be accurate or contain enough information, however, when run through the function it takes a string like this:

Formal Approval for Invoice Number: 996782431 – Amount Payable: $22,632 – From: Company A – Account: bb1a3d0f52fa761fc48eb16c9fd8eb02385c8e70 – To: Company B – Account: 6044ca16e88a572900871b4f3ce791f710f6072f

And turns it into a hash like this:

0xc30a6ecc1af1fb39adeb443914cc0a4bccd4a31169048dc7bdb9ceb536b5c9cf5282b156b2f84defdb6d35e902fda4e9c5b14ca570202e4b70cd8e1fc3ad2a9c1c

Which can then be decrypted using the buyers public key for verification that the invoice was in fact approved by the buyer.

Front-End Work

In contrast to last weeks Back-End focus, this week has been all about getting information from the back end to the front end, making sense of it and displaying it in a clear format, as well as cleaning up my “testing” functions and turning them into proper functions.

Listing Invoices

After last weeks progress of being able to publish invoices to the Blockchain and get basic details back into the console, it was time to get those details into the front end.

The first step was to get the list of invoices, I assumed that this was going to be a trivial task as I was already able to get the details of each invoice individually, so I assumed I would be able to run a for loop and get all the invoices, then I would just need to format and display it, however, I had to do some additional learning about JavaScript Callbacks and Promises as well as Synchronous and Asynchronous programming.

The problem was that when the code ran, it would not wait for a response from the Blockchain before continuing, this meant that variables were left undefined while the code attempted a concurrent call to the Blockchain (which it doesn’t handle well). After some research I figured out how to get the code to wait for a response and eventually, ended up with a list like this:

1 Listing Invoices

Listing all Invoices

Success! Next, it was on to displaying all the details in my invoice template, again, this was not quite as simple as I first anticipated.

Because I was already getting all the invoice details I figured that I could simply pass the variables containing the details to a new window and use AngularJS to bind them to the various elements of the invoice template.

As with many things, this was easier said than done. The hardest part was sending the variables to the template window, however, after some further research I managed to do it using some more Angular magic ($window.open) and spent quite some time working with the bindings. The end result:

2 Invoice Template

Invoice details retrieved from the Blockchain

A fully functional, dynamic HTML invoice template. If only my descriptions were more… descriptive… 😉 Now that I could open the invoices, I needed to add the basic accounting functionality (signing and paying).

To do this, I added a small options card to the side of the invoice template which has different states depending on the invoice state, and will eventually change depending on whether the active account is the “supplier” or the “customer”, but small steps for now!

This slideshow requires JavaScript.

Goals for Next Week

Now that I have the invoicing system mostly up and running, I want to focus of using it from the different perspectives, currently it is very much in a testing type scenario where I am only using one Blockchain account to act as both the supplier and customer (to themselves) so I would like to alter this so that I can switch accounts and filter information depending on the active account.

Also, I would like to start working on the “Make a Payment” page of my app, where one account can send payments to another and include classifications for the payments.

If I manage to get all of that done, then I would like to start working on getting lists of transactions for the accounting side of the app.

According to the planned schedule in my proposal, I was aiming to start populating the Blockchain with test data next week as well. This will all be dependent on my progress and may need to be pushed back to the following week.