Implementation and integration product catalog
Project description
The customer - a Germany-based telco company - wanted to significantly improve its time-to-market and its implementation efficiency for product launches. For this purpose, a central product catalog should be implemented, which should replace the inadequate and redundant mapping of landline/DLS broadband and mobile communications products in the legacy systems for processing customer orders and maintaining customer contracts (CRM).
The project pursued the vision to enable the telco-business to introduce a new product portfolio without any system changes across all order channels and across all brands. Instead, adjusting the configuration in the newly developed product catalog should be sufficient. Product management should be enabled to carry out such changes in the product catalog itself one day. At the same time, the provisioning of orders should come as close as possible to “zero-touch” processing .
The diagram below illustrates the target architecture associated with that vision:
Explanation of the target architecture:
The product catalog as a central database with the products, including their properties as well as the entire declarative logic for the products
The product designer as a UI based application through which the product manager defines the products according to market needs
The product configurator which contains all executing product logic; which executes the product logic depending on the situation:
whether it is an existing customer or a new customer
which business case it is (new product, product change or product change (contract adjustment))
which channel the order comes through
whether any promotions currently apply
in the case of broadband products: which technologies with which bandwidth (line attenuation) are available at the address
Each sales channel was supported by independent systems (portals). The product configurator should be integrated everywhere and replace the product order logic there. Because the system was too outdated for shops and sales partners to continue to use it sensibly and efficiently in the new architecture, it was decided to replace it with a completely new development and to implement it as part of this project.
In the billing area, a new system was also implemented parallel to our project. This pursued a new concept, also with the aim of significantly reducing the time-to-market. It was based on the fact that product price information should no longer be maintained here. The new product catalog should take care of this and provide the price information together with the contract information to billing when contracts are concluded.
During the course of the project we faced another task:
A parallel project was currently in the process of introducing a new document management system, which would also be integrated with the front-end systems (portals) for the various sales channels. In this context, it was decided to additionally implement a document catalog in our project and integrate it with the document management system and the front-end systems (portals). In a similar way to new products via the product catalog, the document catalog made it possible to define which documents are sent with which product order, from which customer group and via which channel. This affected general terms and conditions, cancellation instructions, product brochures, etc.
My role and tasks
As part of a program designed to restructure and consolidate the company's system landscape, I planned the project and managed it using agile/hybrid methodology.
When the project architect left the company after almost a year, I took on a large number of conceptual tasks.
I performed a huge amount of hands-on business analysis; Of the approximately 200 user stories created in 2014, created half of it.
With the support of the head of the above program, I put a lot of effort and energy into convincing all areas of the company of our vision and approach. This affected the management, product management, sales, order management, customer service, the technical area and, last but not least, the IT department itself.
Finally, my task also included planning, organizing and implementing training for the entire sales organization with around 1,000 internal and external employees.
Results
After about 3 months, the product catalog including the product designer was completed as an MVP and made available to the product management department.
The product catalog was integrated with the product configurator, which in turn was integrated with the customer self-service portal to enable ordering of mobile products for the company's approximately 600,000 customers under three different brand identities.
The sales figures via the Customer Self Service Portal multiplied in the following months.
The time-to-market for new product launches for the Customer Self Service Portal was reduced from several person days/weeks to a few hours.
After just another six months of development, pilot operation of the new sales portal for shops, sales partners and call centers began.
A few months later, the first sales region was rolled out.
Productive operation showed no issues of significance.
A training program for over 1,000 sales employees and external partners was created and implemented.
The most important thing: The feedback from users in sales about functionality and user-friendliness was outstanding and characterized by tremendous exitement!
Key figures & facts
The core of the project team consisted of around 15 people + 4 product owners, all of whom were heads of different sales areas
The project duration was 2.5 years.
Performance in numbers:
4,689 PD (person days)
515 user stories
51,493 Lines of Code (LOC)
12 business cases across mobile and broadband productportfolios
System environment: Oracle, Java, Tomcat
Tools used:
MS Office
MS Project
Jira
Confluence
sonar
Gliffy
Balsamic
FishEye
HP ALM/Quality Center
HP Service Virtualization
SAP
Project Organisation
The core team of the project included the following members:
1 project manager (also took on the role of Scrum Master)
1 business analyst
2 architects with 50% each, who also helped develop it
5 backend/enterprise developers
2 front end developers
1 test manager about 50%
1 functional tester
2 test automaters
2 representatives from order management / testers each with around 50%
There were also four business product owners:
Head of Customer Self Service
Head of Online Sales
Head of Shop Sales
Head of Sales Partner Sales
Methods and tools
Circumstances and conditions
The project broke new ground in many ways; the concept of catalog-based product ordering, agile development, project-based work were all new approaches for the organization. Our project was therefore faced with several challenges, which I will discuss in more detail in the following sections.
New concept, new ubiquitous language
For the customer in general, this project meant the adoption of a completely new concept; that of catalog-based order processing. Over the years, the entire company had gotten used to the technical conditions, which were characterized by the limitations of the existing systems. Thus, an “IT technical reality” and, above all, language developed that was largely far removed from the technical reality. The department had also become accustomed to this language over the years. As a result, discussions about technical requirements and the acceptance of the new concept became significantly more difficult. A lot of effort therefore had to be invested in “picking up” the entire organization and, above all, product management.
Two speed IT
Our project aimed to create a new system world in large steps. It was clear to all of us that the challenges were enormous. There would also be problems that we had to be willing to accept. We firmly believed that high speed of implementation and willingness to take risks were essential to achieving our goals. This also fit well with the culture of the sales departments, our primary stakeholders.
However, this approach collided - understandably - with the culture in the IT department that was more focussed on problem-avoidance and process conformity.
Architecture
At the start of the project there was an monolithic system covering
Customer management
Contract management
Order management
Product management
Technical provisioning (setting up configuration in the technical systems, for new customers or customers who made a product-change
A large-scale program aimed to break this system down into several components covering the various business areas. In this regard, our project primarily covered the areas of product and order management. The company's architecture group tried to model the future system architecture using the SID model from the TM Forum.
Approach
The development was completely based on Scrum. The project was embedded in a program that was classically structured; with project managers for various projects who created time and budget planning and reported on them weekly. Agreements on important milestones had also been made with the primary stakeholders. The entire procedure can therefore be characterized as a hybrid.
Business Analysis
Initially, our business analyst created detailed EPK diagrams during the technical design, accompanied by descriptions based on the eTOM framework from TMForum. However these didn't provide enough support for developers; the diagrams were very difficult to read and despite (or perhaps because of) their enormous size, many key questions remained unanswered from the developer's perspective.
As a result, we brought the technical concept to two new pillars.
Following the principles of Domain Driven Design (Eric Evans), we created technical models. The team designed the technical data models for all areas together and - this is the crucial thing - discussed them intensively with the departments. These models were created using UML without claiming to be 100% methodologically correct. The aim was simply that the essential questions could be answered in discussions within the team and with the departments. This resulted in a so-called TK domain model, which represented a uniform definition for terms and entities across system boundaries. This should then later apply to the entire company.
For all masks we created UI mockups using the Balsamic plugin for Confluence. We then described all the essential business logic for the masks in Confluence.
We created user stories that were essentially based on the masks. There were several user stories for each mask, which were linked to the respective mask on the Confluence page.
We discussed the above-mentioned technical models, the mask designs and the user stories with the manager responsible for sales in a product owner group, which proved to be extremely effective. Backlog prioritization was also discussed here.
Below is an example of a technical model that I created to design user management and the administration of sales points (own shops and sales partners).
We had an interesting discussion in the project as to whether such a “technical” model could be discussed with the business or not. We just did it and it worked wonderfully.
It took us a few rounds to finally agree on a final version. The subsequent implementation was quick and smooth.
Development
Development was based on Scrum; Pair programming was used, there were peer / code reviews with the help of FishEye. There were groomings, planning including planning poker, 2 or 3 week sprints and sprint reviews. There was continuous integration and a maximum level of test automation was carried out through development (unit tests). This was monitored with Sonar reports, that were continuously displayed on a large screen in one of the developer offices. Furthermore, CVS, GIT and Jenkins were used in development.
The models that were created for all functional domains (see above under Business Analysis) enabled clean development which led to a software stack that was very adaptable. A effective set of development guidelines was established and rigorously followed. Because of this and because it was easy to recognize business logic when reading the code, it was also relatively easy to integrate new developers into the team.
The mask designs, which were consistently created and discussed with the product owners in advance, allowed for rapid and efficient development because the departments had already “seen” the application before it was completed. Big “aha” experiences followed by extensive re-factorings were herewith avoided.
Test
The test team used HP Quality Center for test and defect management. For development defects were transferred to Jira via an interface. Within the Jira tickets one could drill down to relevant code fragments. In addition, HP Service Virtualization was used for automated tests at the integration level. These tests were also extremely helpful in getting to the bottom of performance problems, because for individual transactions or calls from the front end one could track which service and which system in the middleware or backend took up which amount of the total response time.
Hybrid Projectmanagement
As described, development was based on Scrum. This however created a challenge of making binding commitments to stakeholders on key milestones. These stakeholders included the program manager, management board of the company, the sales department and customer service. They needed our commitments for planning sales targets as well as for personnel planning. And on their turn those stakeholders needed to give commitments to the management board of the mother-company.
In this regard, I proceeded as follows:
I occasionally exported the backlog planning from Jira to Excel.
Because only the user stories for the coming sprint were groomed and pokered, I initially had no reliable indication of when the user stories would all be ready. that were needed to complete a specific increment/milestone
I then added columns for different implementation areas (Front-end, Backend) in Excel. SML complexity assessments were then made for these areas for each user story by the architects and developers.
The effort per developer discipline was then calculated using an algorithm.
The efforts that were determined in this way were calculated against the planned capacities, taking into account vacation times, etc., and then made it possible to determine the dates for the milestones.
Technical concepts
Architecture
The attempts by the companies architecture community as described above to model the future system architecture using the SID model from the TM Forum were of little to no help to the developers in our project because they were far too abstract. To close the gap, I created my own project-related architectural diagrams using the Gliffy plugin for Confluence and the software architects in the project team then helped me to correct them .
These images helped us to effectively discuss architectural questions in the project and to make important decisions effectively. In addition, it offered effective support for developers during implementation.
In addition, these images helped me as projectmanager to derive a “roadmap”. Of course, we had to think about the steps in which we could achieve our target architecture. These steps then became project increments that resulted in a roadmap (shown in the image below).
For each of these increments I created my own version of the architectural diagram, which showed in color which parts were finished by then (green), which were in progress (blue) and which were still open (yellow and red).
This approach turned out to provide very good and understandable transparency in stakeholder meetings. This way everyone could see very clearly how far we had already progressed, through which stages (with which steps) and also that we were pursuing a realistic approach. That created the trust we needed to be able to continue..
The image below shows an example of such an architectural increment-diagram from the beginning of the project:
Catalog driven customer ordering
An elementary task in the area of business analysis and architecture was the design of the product catalog-based ordering - primarily the design of the interaction between the product catalog and sales portals and downstream systems that were responsible for order processing and setting up the technical systems i.e. the provisioning.
The core principle of the product catalog was the use of hierarchical relationships:
this item belongs to this product
this element consists of the following elements
The following technical components are required for the realization of this commercial element.
There were different types of product elements, such as product options (a specific router or an installation service), discounts, etc. All product elements had different properties, such as a description and a price. Cardinality was used to determine whether an element is a mandatory element or optional, as well as whether single or multiple selection is possible. The structures created in this way formed so-called product trees. Rules could be defined on individual elements within a product tree, for which there were also different types, such as product-switching rules. The entire declarative product logic for products was mapped via these structures and made available to the product configurator.
For example, a product tree for a fixed-line DLS broadband product looked something like this:
The creation of these structures - product modeling - was carried out with the product designer; in short, the UI for the product catalog. The product-configurator, which was integrated into the sales portals was able to interpret the product-trees handed over by the product catalog and apply runt-time logic on it based on the real cricumstances:
Channel
Brand
Customer type
Use case
For broadband: Technical feasability
So this way new product portfolios or adjustments existing ones could be done via the UI without system adjustments in the sales portals! This led to real time-to-market advantages!!
Servicecatalog driven provisioning
In order to fully exploit the time-to-market advantage, not only customer ordering, but also technical provisioning should be catalog-based. Therefore we created a so-called service catalog. This contains technical services for the factual and technical realization of the commercial products. Basically references for system in OSS to execute the proper configurations, for logistical system to provide proper shipment of devices and references for commercial partner systems like for example Vodafone.
For example, an FN/DLS broadband product requires a line that can be provided using various technologies. These technical services are related to product elements of the commercially defined product tree, but are not part of this tree.
The image below visualizes this approach.
This should make it clear that with such an approach, new products (product portfolios) can be introduced in an almost completely configurational manner with a corresponding time-to-market advantage - even if nothing changes in the landscape of the technical systems.
Advantages of catalog-based ordering
The legacy system knew neither pure products nor pure technical services. Products were multiplied by technologies available, so there were as many instances of a product as there were technologies through which it could be realized. This seems logical at first, but it has a glaring disadvantage: product change processes become very complicated.
This can be illustrated very easily using an example with two products and four technologies. The following image visualizes the situation when products are multiplied by technology.
Instead of having only two product and one relationship between then there are eight products, with the following consequences:
As part of the product change (switching) logic in the system, 16 relationships must be maintained between these products. Across all products - not just the current one, but all product generations (including inactive products) - this becomes an extremely large number of relationships.
Technical logic for a technology change and business logic for a product change are mixed and this matter-of-fact combined with the huge amount of interrelationships, is very extensive, very difficult to understand and very difficult to maintain.
Provisioning logic must connect to many products rather than a few technologies.
The result:
Changes, especially changes in technology, result in major adjustment efforts. As a result, this approach has very high costs and very poor time-to-market.
In contrast, the image below visualizes the situation when products and technologies are clearly separated.
The depiction of products in the system corresponds to commercial reality; there are only two products.
The technologies are related to the products, but are not part of them
As part of the commercial product-switching logic, only a single relationship needs to be maintained
This relationship is so simple that you get into the area where changing the product-switching logic via the product catalog becomes possible without any system-changes
Business logic for determining the technology to realize broadband connections for individual customer settings as well as for their provisioning is much leaner because it is not multiplied across the products and is therefore much easier to change
The provisioning logic for a technology change for an individual customer, which may accompany a product change, is also better manageable. Because here too there are only a few relationships.
So all in all:
This approach which was taken by the project results in a much leaner system, which corresponds much more to the commercial and technical reality and is therefore very understandable for the developers. As a result, adjustments are made much faster and are therefore much more cost-effective. There are also product changes that can be made without adjusting the system. It no longer needs to be mentioned that this leads to a significantly better time-to-market.
Technical feasibility check
A new technical feasibility check (the requirements of a house connection in order to determine the maximum achievable bandwidth and possible products) had to be implemented in the sales portals - analogous to the technical feasibility check in the existing legacy system.
For this purpose, I created the process model shown below. The logic is shown here as to which technology a connection can be implemented. It determines which technology is preferred when there are several options, taking into account the line attenuation values.
As can be seen, the process model contains several “if/then” conditions and, based on them, several loops. Designing such a model was a conscious decision, because it was very close to the programming paradigm and was still very comprehensible for the colleagues in the business departments involved. Before implementation, they were able to easily verify whether our understanding and approach were correct or correct the rules.
The subsequent implementation was extremely efficient: the developer completed the development in just two weeks. It was able to be fully tested within one release and introduced into production without any problems.