I recently participated in a new project of the company. I need to synchronize
接入方the data, such as enterprises, orders, contracts, logistics, etc., to our platform through the openapi interface , and then our platform provides them with financial capabilities.
对接方is not the same city, in order to improve efficiency, the two sides conducted a number of online video communication. At first, it went smoothly. I didn't expect that when communicating the corporate information upload interface, there was a very inconspicuous
企业注册地idfield in the interface document , which made us suddenly enter a deadlock.
What exactly happened?
1. Regional issues
企业表There is a
企业注册地idfield in our platform , which is required. The user needs to select a region on the page of registering a company as the registration place of the company. In fact, the database saves the ID of the region.
If the company is successfully registered, the area name will be displayed on the company details page. Of course, the background logic of our system is to first pass
地区表anti-detection of the region name, and then display it in the user interface.
企业表to be consistent, when we define the interface document, the id field of the company registration is also made mandatory.
The situation at that time was like this: Our region table had fields such as id, region name, national standard code, grade, etc., but the id here is the primary key of our database, which is definitely not available in the docking system. The counterparty system also has a set of region tables, but the id is their database id, and their tables also have fields such as region name, national standard code, and level.
So they need to go through some conversion within the system to pass the area id we need to us.
1.1 Persistent local region table
In fact, I joined this project halfway through. I was dealing with other things before, and the interface document was already defined when I joined.
When we communicated with the counterparty for the second time, the two parties reviewed the details of the interface document together, including: the role of the interface, the meaning of each parameter, and whether they have a value to pass, and so on.
When the enterprise information upload interface is
企业注册地idpassed , there is a field in the interface document, and the other party cannot pass the value. In order to solve this problem, our first version of the plan is:
- The docking party calls our region query interface, and through multiple paging queries, we can finally obtain all our region data and transfer it to their local region table.
- Before they call our corporate information upload interface, they first query the local area table and convert it to the area id we need.
During the discussion, the counterparties felt that they were also platforms and should not do these additional things. So in that meeting, the two sides did not persuade anyone on this issue, and ultimately failed to reach a consensus.
Later, I thought about it, and it was true that this plan was too idealistic. It didn't really think about the problem from the user's point of view and ignored many details. It may be related to the document designer who is not familiar with the regional table.
1.2 Call the area query interface by name
During that meeting, some of our colleagues discussed briefly. Since the docking party is unwilling to accept the persistence of the regional table in their local area, we will put it second and do not require them to persist. At this time, a colleague of ours proposed to call the area query interface by name and find out the area id. The specific plan is as follows:
This solution seems to be no problem on the surface, but I have been responsible for regional related functions before. I know, I am afraid that the following situations will occur:
- If the name of the docking area square pass is incomplete, for example: originally
成都市, the actual upload
成都. In this way, we need to do fuzzy matching for querying the interface in our region. If the interface is called concurrently, it may affect the performance of the interface.
- If you enter a keyword
北京市, you can find two pieces of data in our area table, one is the
省级别same, and the other is the
市级别same. Which data does it correspond to?
So I raised these two questions at the time, and it is not recommended to use the area name query.
1.3 Call the area query interface according to the national standard code
After listening to that colleague, he also felt that it was a bit unreliable to query by district name. He immediately modified the plan to use the national standard code of the region to query the region id. The specific plan is as follows:
Since the discussion time was very short at that time, we did not have time to think about it too much, so we plan to use this plan for the time being.
1.4 Enterprise upload interface to enter the national standard code
After a while, the two parties continued to review the interface document and re-discussed the issue of
企业注册地idfield transfer in the enterprise information upload interface .
Before they adjust the corporate information upload interface, first adjust our region query interface to find out the region ID, and the entry is the national standard code. Then pass this area id in the enterprise information upload interface.
The counterparty listened carefully to our plan and hesitated. They felt that there was no need to adjust the area query interface again. Shouldn't both parties use the national standard code?
Their idea is: In the enterprise information upload interface, the entry parameter is
企业注册地国标码. Since the national standard code is the only code unified by the country, the two parties must be the same, which can ensure the consistency of the data.
2. Remembered a question
To be honest, if you have not been exposed to regional functions, most people may agree to this plan.
But it is more coincidental that I happened to be exposed to similar functions before. At that time, I suddenly remembered a question: How to ensure the consistency of the data on both sides?
We all know that due to the development of the country, some cities may be renamed, such as:
襄阳, and sometimes multiple prefecture-level cities are merged into one city, so the national standard code will change, so the national statistics network will adjust the regional name and National standard code.
Our regional table was created two years ago, and it was updated after the data was initialized.
The docking party does not initialize the data at the same time as us, and they will regularly update the regional data, which leads to inconsistent data on both sides. If the newly added city name and national standard code are used in the business form of the counterparty, and the information is not in our region table, the region id we need cannot be queried.
What should I do in this situation?
2.1 Both parties update the region table at the same time
Obviously the above problem is a very difficult problem. At this time, some friends may say:,
双方使用job同一时刻更新地区表can the problem be solved?
I am not in favor of this scheme, the main reasons are as follows:
- We only have a synchronous job with this counterparty, no problem. But if there are other docking parties, they also need to call the corporate information upload interface. Is it necessary to have a whole job, and it also requires everyone to execute at the same time, and the coupling is too great.
- If our party and the counterparty execute the job at the same time, if any party fails to execute the job, it will also cause data inconsistency. If the docking party is calling the corporate information upload interface at this time, will there be a problem?
2.2 Subject to the regional data of one party?
The above scheme of updating the regional table at the same time by both parties is indeed a bit unreliable, but some readers may ask that the regional data of one party shall prevail, and the other party will not be able to synchronize the data. The specific plan is as follows:
This plan is actually very similar to the first plan given by our side, and has been rejected by the counterparty. From their point of view, there is really no need to upload corporate information and save our regional data.
To be honest, even if they agree, this kind of cross-company and cross-system data consistency problem is not easy to guarantee, because if the docking party fails to call our regional interface, at this time, it happens to upload corporate information, is there a problem? ?
We were in a dilemma at once, but in order not to affect the overall progress, we can only record the problem first, then skip this problem and continue discussing other fields.
3. How to solve this problem?
I thought about it for a long time that night, and the next morning, I found that it coincided with the idea of our boss. The conclusion is that since there is differentiation, and there is no way to avoid it, we have to accept differentiation from the system design. Two fields are added to the enterprise information upload interface:
地区名称, the docking party changes to the incoming two fields. The specific scheme is as follows:
- Adding the area name field to our company table is not required, and at the same time, the previous area id field is also changed to not required.
- When the docking party calls our corporate information upload interface, it also inputs the regional national standard code and the regional name.
- It is judged in our corporate information upload interface that if the area id can be found by the national standard code, the area id will be written into the db, if it is not found, the area name will be written into the db.
We evaluated the scope of influence. The region field in the enterprise table is only for display purposes without modifying the entry, so the above scheme is feasible.
Later, when we communicated with the counterparty online again, we told them about our plan, and they very much agreed.
Although this regional issue is not worth mentioning among the many technical issues. But I thought about it carefully, and there are still some valuable experiences worth summarizing, and a reference for those in need.
4.1 Design the interface from the user's point of view
When designing interface documents, we must truly proceed from the user's point of view.
Especially for this kind of openapi interface, the defined parameters should be as common as possible, and everyone recognized parameters, to avoid our customized parameters, such as region id.
Try to reduce the complexity of users and make it easier for them to call the interface.
4.2 Technical solutions must be inclusive
Technical solutions must be inclusive, not black and white, and flexible thinking. In a distributed environment, if you blindly pursue strong data consistency, there will be no good results. Just like the commodity spike system under high concurrency, if you have to use a synchronous solution to implement it, the system may eventually hang. A better solution is actually to change to asynchronous queue processing.
Both our side and the counterparty have regional tables, and it is difficult to guarantee complete data consistency. We should not use consistency for consistency, as this will be counterproductive. In order for the work to proceed smoothly, one party must compromise. My suggestion is for the openapi interface party to compromise. This kind of technical solution is universal.
4.3 There is no best solution, only the most suitable
Our last plan did not completely solve the problem of not finding the region ID, but from a business perspective, even if there is no region ID, the region name is the same. Obviously, the final solution is very suitable for our actual business scenarios.
Therefore, there is no best solution, only the most suitable for business scenarios.
4.4 Conduct effective communication
When communicating with the counterparty online, don't stay stuck because of a problem. If there is no good technical solution at the time, you can choose to skip this question for now and communicate other content. Later, we will spend time alone in private to think carefully about the problems at the time, so that we can come up with more reasonable plans.