Installer and upgrader rewrite

[Sesquipedalian]

@jdarwood007, let's use this to work collaboratively on the installer/upgrader rewrite.

• [x] Add Schema for 3.0
• [x] Add Initial framework for Maintenance logic
• [x] Build Installer logic
• [x] Setup Migration logic for upgrades from SMF 2.0 to 2.1
• [x] Setup Migration logic for upgrades from SMF 2.1 to 3.0
• [ ] Test Installer logic
• [ ] Build Upgrader logic
• [ ] Test Upgrader logic
• [ ] Build Converter Logic
• [ ] Setup Converter logic for YabbSE
• [ ] Setup Converter logic for SMF 1.0
• [ ] Setup Converter logic for SMF 1.1
• [ ] Build Repair Settings tool
• [ ] Build logic for self-contained tools to utilize the maintenance logic.
• [ ] Add logic back for installer to work with compression (?obgz)
• [ ] Allow upgrader to support login or entering in the database password.

Fixes #8261

[jdarwood007]

A lot is incoming, it's my initial work I started to write the installer logic. You should be able to get through an install. But the base logic should get it going for an upgrader and converter logic.

[jdarwood007]

I've also added some task notes based on what I was thinking, please amend or adjust.

[Sesquipedalian]

Please remember not to force-push to a shared branch, @jdarwood007. I was able to reconcile the differences in my local repository fairly easily this time, but another time I might not be so lucky.

[jdarwood007]

I will try to remember that. Would you prefer a merge then? I was bringing it up to the current 3.0 branch.

[Sesquipedalian]

I will try to remember that. Would you prefer a merge then? I was bringing it up to the current 3.0 branch.

For that situation, yeah, a merge would be the better option.

In general, I'm just asking that you don't amend commits after they've been pushed to this branch. Feel free to amend commits and rebase stuff as much as you like in your local working copy before you push them. That's cool. But once a commit has been pushed to this shared branch, it shouldn't be changed.

[jdarwood007]

Will do merge then. Rebase wants me to do a force push to get it into sync.

[jdarwood007]

@Sesquipedalian Minor detail, the check script check-smf-license.php fails, because its looking for SMF_VERSION in upgrade.php. I'm wondering why this file was checked instead of index.php. Any issue if we change that?

[Sesquipedalian]

None at all.

[Sesquipedalian]

In my tests, the installer now works nicely.

However, I have not tested on a Windows server (and cannot do so). @jdarwood007, if you can confirm that the installer works on Windows, we can mark off that task on the checklist.

[jdarwood007]

In my tests, the installer now works nicely.

However, I have not tested on a Windows server (and cannot do so). @jdarwood007, if you can confirm that the installer works on Windows, we can mark off that task on the checklist.

I will have to setup an instance on my windows box but can. I still need to get through upgrade process and confirm it works. Will do it after that since we should just have conversion logic to build for YabbSE/SMF 1.x. After that its cleanup and getting CLI working (I'm adding code but not testing it yet).

[Sesquipedalian]

FYI, I removed the $initial_columns property from SMF\Db\Schema\Table and all the table classes that extended it. Between the data in the $initial_data property and the column definitions in the Column objects, we already have all we need.

[jdarwood007]

@Sesquipedalian Might be all the time I have for today, but I added some initial logic to show how the upgrade processes.

Already ran into an instance where a query is not both MySQL and PostgreSQL safe. But others I was able to make it safe. I haven't tried to execute the code yet, just working on converting the steps from .sql files into a .php file.

[Sesquipedalian]

I made some changes, @jdarwood007.

First, I moved some stuff around and adjusted some names to make naming and organization more consistent.

Second, instead of using numbered migration sub-steps, I gave them all names and created an array constant (SMF\Maintenance\Tools\Upgrade::MIGRATIONS) that lists them all in the intended order. I did this because (1) it is more reliable to use an order that is explicitly defined in code rather than an implicit order that depends on number file names, and (2) this will make it easier to adjust the order whenever necessary. The latter consideration will become important once we start introducing more database changes as 3.0 develops.

[jdarwood007]

That works as well. My intention for the numbering was to just glob the directory, which then with them named in an order, would return a list for us to use to iterate through with an iterator seek (https://www.php.net/manual/en/arrayiterator.seek.php) when we restart the script.

I have had no time for coding lately. It spring time, so house work to be done and I'm releasing a new app for work. Trying to take time off, but its going to be for more house work.

[jdarwood007]

I lost the last half hour of work I did to custom fields. Will try to work on this more as I get time. I am currently in the 'Messenger fields' section in the SQL files. You have to compare both MySQL and Postgresql because we appear to sometimes do different things or skip steps in some. The SQL files are still about 2500 lines of code to review. We still need to build converter logic as well to handle YabbSE, SMF 1.x, and SMF 2.0 (if we don't want to rewrite its upgrade). As well we will have to completely revalidate the entire 2.0 to 2.1 upgrade logic here.

[Oldiesmann]

Found an install-related bug that should be fixed in this PR assuming it still applies (I haven't looked at the code yet). The MessageFormatter stuff relies on $txt['lang_locale'], which is defined in the index language file. However, this file isn't loaded in install.php, which will cause errors. Normally I'd just submit a PR for the fix but since you're in the process of rewriting the installer anyway I didn't think that was necessary.

[jdarwood007]

I know ICQ was removed, but the logic was for 2.0 to 2.1 upgrade. Shouldn't it be preserved? Let the 3.0 upgrader remove it later?

[Sesquipedalian]

You are correct. I will revert that change.

[Sesquipedalian]

"Fix 2.1 schema namespace"

Wow. I can't believe I missed that. Good catch!

[jdarwood007]

I didn't either until I remembered I was writing the upgrader for 2.0 to 2.1 and I was referencing 3.0 for the schema. Tried to use 2.1 namespace and my editor complained. Tracked it down to that.

[jdarwood007]

Finally got through the 2.1 upgrader sql logic. What a mess that was.

I started on the migration logic itself.

I also moved the Maintenance.php from Sources to the Maintenance folder. Had to fix some references for that. If we don't want it there, we can move it back.

I also deleted the upgrade .SQL files for both 2.1 and 3.0. It will make a merge conflict if we make changes, but forces us to find and fix anything. I put the install.php in place. You can swap out install for the upgrade call to see it. I didn't want to overwrite upgrade.php until I have finished using it for referencing while building the migrations,cleanup and done steps to the upgrader.

But I think having the install.php will let us fully test the installer to make sure we are happy with it and can cross that off the list.

There is a note where I left off on the migration logic I am working on for the upgrade itself. I need to get back but wanted to get loaded what I have so far.

[jdarwood007]

@Sesquipedalian I'm running into an error: ( ! ) Fatal error: Uncaught Error: Typed static property SMF\User::$me must not be accessed before initialization in /Sources/BrowserDetector.php on line 131

Its being called in the Migration/v2_1/BoardDescriptions

					'name' => Utils::htmlspecialchars(strip_tags(BBCodeParser::load()->unparse($row['name']))),

What is the optimal call here to put the best effort that SMF can do to safely startup the User::$me object? Keeping in mind we may be coming from a env in which it may not be able to startup things.

[jdarwood007]

		// Load up the current user safely.
		$p = User::load(0);
		User::$me = $p[0];

This seems to call ok.

[Sesquipedalian]

		// Load up the current user safely.
		$p = User::load(0);
		User::$me = $p[0];

This seems to call ok.

This would be better:

		// Load up the current user safely.
		User::setMe(0);

Also, is it indeed the intention here to load a guest user? Because that's what loading user 0 does.