[Bug] database is null when quering

[Z-SWEI]

Search before asking

• [x] I had searched in the issues and found no similar issues.

Version

3.0

What's Wrong?

In function CollectRelation.collectMTMVCandidates, new BaseTableInfo(table) may cause exception "database is null", because ExternalTable.db will never be set for FOLLOWER FE loads from checkpoint image.

the exception occurs: DatabaseIf database = table.getDatabase(); // for external table it may be null, when it is from image, and db is null java.util.Objects.requireNonNull(database, "database is null");

What You Expected?

check when collecting mv for external table ?

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• [x] Yes I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[Z-SWEI]

Hi @seawinde , I noticed this issue may be related to the changes introduced with collectMTMVCandidates. When you have a moment, could you please review whether this part of the code might require an update?

What I’m thinking is that there are two possible approaches:

1. Handle ExternalTable specially in collectMTMVCandidates.
2. Set the DB when loading ExternalTable. I’d appreciate your feedback. Thanks!

[seawinde]

I believe that, if the root cause is that the FOLLOWER FE loaded from a checkpoint image never has ExternalTable.db set, then the error occurs precisely because this field is unset. Therefore, it would be more reasonable to explicitly set ExternalTable.db. and what dou you think? @morningman @zddr @Z-SWEI

[morningman]

which doris version are you using? and please show me you show catalog xxx result

[cai-jinlin]

Root cause of the anomaly: When a FOLLOWER node recovers from a backup file, the "home" information of external tables (such as databases like MySQL or SQL Server) is lost. This causes the system to fail in determining which database the table belongs to during subsequent query optimization, resulting in an exception. Essentially, this is an "information loss" bug during data recovery, leading to system crashes when external tables are used in subsequent operations due to empty values.

Solution: When restoring an ExternalTable from a backup, immediately locate and set the correct database reference based on the database ID in the backup. The backup file already contains the database ID, but the original code did not use this ID to find the Database object and establish the association during reading.

The benefits of doing this:

1. Once set up, all subsequent users benefit. No need to write redundant code or repeatedly check if the database tables are empty. You can immediately establish a reference from the table to the database. The backup file already contains the database ID.
2. Backward compatible, does not affect normal processes, only fixes exceptional cases.

[cai-jinlin]

// 添加数据库引用恢复逻辑 Status ExternalTable::read_fields(DataInput* in) { // 先调用父类方法 Table::read_fields(in);

// 读取数据库ID
int64_t db_id;
in->read_int64(&db_id);

// 核心修复：根据ID找到Database并设置引用
if (db_id != -1) {
    _db = Catalog::instance()->get_db(db_id);
    if (_db == nullptr) {
        LOG(WARNING) << "Cannot find db " << db_id << " for table " << _name;
    }
}
// 继续读取其他数据...
return Status::OK();

}