[CE 2.1]
The project “Designing of Distribution Storage Solution” was completed at Huawei Technologies Co., Ltd.
| Project Title | Designing of Distributed Storage Solution |
| Duration | 8th October 2012 – 25th January 2013 |
| Location | Beijing, China |
| Organization | Huawei Technologies Co. Ltd. |
| Position | Engineer |
[CE 2.2]
Since 2011, distributed storage became more and more popular in the storage market. Compared with classical storage products, it stores data on a multitude of standard machines, which behave as one storage machine although data is distributed between these machines. It provides a better cost-performance ratio, flexibility, and scalability than those of classic storage. Unfortunately, at that time Huawei did not offer any distributed storage solution. Huawei would not miss the huge business opportunities to access the market of distributed storage. Therefore, a project was set up to deliver a distributed storage solution for one very important customer.
[CE 2.3]
The overall aim of the project is to deliver a competitive distributed storage solution in the technical, environmental, commercial, and legal contexts. The detailed objectives of the project are provided as follows:
[CE 2.6]
I led a project team with three engineers to deliver the necessary systems. Those systems would be integrated with other systems developed by other teams into the whole distributed storage solution. My duties in the project were presented as follows:
[CE 2.11]
Considered offering low initial investment, I chose standard x86 server as the underlying platform on which the monitoring system ran instead of Huawei's dedicated storage server. I summarized the whole solution as a design specification.
[CE 2.12]
Problem: There was another challenge I applied my engineering skills for fulfilling the requirements on data distribution. I made sure to evenly distribute the data stored in the storage systems. I proposed a layout system based on a random process to resolve the issue. I defined the available capacity of one machine as the weight of the selection of data layout. During the process of choosing data layout, I summed up all weights of the machines in the cluster, and then I made the sum modulo which was a random sequence to determine the location of the data. The more capacity that one machine has, the higher the possibility that it might be chosen. I implemented a quick prototype using MATLAB and verified its effectiveness.raised by the colleague that I assumed a fixed server with the largest rank number as the leader and it would be failed while such server was dead. I carefully considered the failure scenario, and I found the risk exactly existed.
Solution: To reduce the risk, I created a new election algorithm to generate a dynamic leading server, so that even though any current leader went down, a new leading server was able to be elected for coordinating a consensus process. I expressed sincere thanks and appreciation to those specialists. They helped me to make the design solution more professional, feasible, robust, and comprehensive.
[CE 2.13]
Among all decomposed tasks, I was responsible for basic functions on which other systems relied. I implemented a Paxos system, random process system of data layout and innovating election system in C language. The quality and schedule of such functions were of paramount significance to the achievement of the team goals. As a result, I designed over 4000 test cases for those systems, and then implemented and executed the test cases. After fixing almost 57 issues, I submitted the systems to the repository. In addition to that, I installed the development environment with three x86 servers; tuned parameters of the operating system, database and middleware for optimum performance; and developed and tested common libraries before respective deadlines. I gained the credit and faith of other members in my team through qualified and timeous completion of my tasks
[CE 2.14]
To meet professional standards from quality assistance team, I took great emphasis on the process improvement and quality management throughout the execution of works of the subproject. I asked team members to apply the refactoring, pair programming, test-driven development, and some other best practices to the tasks of the subproject. With our joint efforts, the project was completed in good time.
[CE 2.15]
I am proud that my team completed the project and contributed to the success of the whole storage solution. Except me, other team members had little experience of a distributed system, but I overcame many challenges through my professional knowledge and skills. Compared with the classic storage product of Huawei, the initial investment of a new distributed solution was reduced by half. This made the solution more accessible to NPOs and small businesses. Also, the scale could be ranging from 3 to 288 machines, and users could easily add machines to the cluster for expanding or remove machines from the cluster for shrinking according to their requirement to the storage capacity and performance. This made the storage solution more flexible and scalable. With the solution, Huawei earned many business opportunities in the enterprise market. To achieve the goals, I clarified and defined customer requirements; proposed and revised the design solution; created an innovative election algorithm; developed and tested core system; conducted management work of the project, and functioned as a leader of the three-person team.
