When the order of fields in the data table is changed, the binlog log is confused.

[ccke]

Hello, when I perform the following steps:

1. Start canal and monitor table A
2. Change the field order of table A
3. Debug binlog to change data and find that data confusion occasionally occurs.

I'm wondering what could be the reason for this? What needs to be done?

[ccke]

Table schema：

Canal table columns：

Canal row values：

[lance6716]

Change the field order of table A

This is done by a DDL? I think you can check if canal has captured this DDL and invalid the internal table structure. I'm not a user of canal interface, but I think it should be caused by table structure mismatch.

[equnchen]

I reproduced the problem.
1.DDL: change the field order => insert => revert the field order => insert

2.Start canal. After call Canal.StartFromGTID(), we will get the newest table schema，but the binlog data can old, results in mismatch of filed.

At present, what we can do is to minimize the canal's master-slave delay，ensure that there is less dirty data in the binlog.

[dfradehubs]

Hello there!

Any news about this? We are getting also these errors in our tool https://github.com/freepik-company/binwatch.

When an alter table occurs, the index of e.Table.Columns and e.Rows[] are not in the same order 😓 .

Thanks!
Daniel.

[dfradehubs]

Okay, it seems the issue occurs when you're trying to read binlogs older than the moment the schema change took place. This happens because Canal directly reads the current schema when it's unable to read it from the binlogs (since it doesn't have that data, from what I understood). Let me know if I'm correct; thank you very much! :)

[lance6716]

Okay, it seems the issue occurs when you're trying to read binlogs older than the moment the schema change took place. This happens because Canal directly reads the current schema when it's unable to read it from the binlogs (since it doesn't have that data, from what I understood). Let me know if I'm correct; thank you very much! :)

I think so. It's hard to get a history table structure in binlog.