[3.0] Adds migration and cleanup steps for upgrading from old versions#9120
[3.0] Adds migration and cleanup steps for upgrading from old versions#9120Sesquipedalian wants to merge 20 commits intoSimpleMachines:release-3.0from
Conversation
Sesquipedalian
changed the title
Sesquipedalian
force-pushed
the
3.0/upgrade_old_versions
branch
from
a2e1ec0 to
ff9e7b5
Compare
|
First attempt at running an upgrade with this PR tonight. First test was a 2.1 to 3.0 upgrade. I had to do a composer update to get this to work. I actually wonder if the /vendor contents & composer lock should have been in the commits??? Not sure what the proper protocol should be. I was able to get past that by committing the update to composer.lock locally in my test branch for this PR. Next problem is that the 'Continue' buttons are all associated with 3 second countdowns. I don't think there should be any countdown at all... That wasn't even enough time to type in my credentials. And it shouldn't continue until I say so... I got past that by updating the upgrade template countdown to 30. And reading and typing quickly. Next problem is that it appears to have automatically started trying to do a postgresql upgrade on my mysql database. Screenshot below. I wasn't able to get past this. Not sure why it's attempting pg, I didn't even have pg started. And the 2.1 style Settings.php file specifies $db_type = 'mysql'; Note that with this screen, it's still doing an auto-continue every 3 seconds: Mysql 8.4.4 |
|
Well, that's a bunch of weird surprises. I didn't touch any of those things in this PR. But I guess we'll need to fix them so you can test. |
I'm not sure why you had to do that. There are no differences between this branch and the release-3.0 branch for composer.lock, nor any for composer.json.
I could have sworn that I removed that in an earlier PR.
That's extremely weird. More investigation is needed before I can even begin to guess why you saw that. |
I believe the latest commit has fixed that one. It appears to have been introduced in #9090. |
Oh, now I see. The screenshot doesn't indicate that the upgrader thought your database was PostgreSQL. It is just showing that the upgrader was walking through some of the early migration substeps that apply only to PostgreSQL (and skipping them because they weren't applicable) when it encountered an error with the message "TypeError: NetworkError when attempting to fetch resource." The question is, why did it encounter that error? Based on a quick web search, it indicates a CORS problem while making an HTTP request, which is rather odd in this context. To help us figure where the heck that error is coming from, please check:
|
|
Good news: I refreshed the code from the latest this morning, & the 2.1 => 3.0 upgrade went well! I found that the composer.lock was simply because I'd run the composer update & there is a new version of fixer available. Side note: I haven't had a successful 3.0 installer run in a very long time (I'd logged several issues early 2025). I still cannot get it to run, with or without this PR. Logged #9131 I just attempted a 2.0 (latin1) => 3.0 upgrade & got the following error. All files are there, I believe, & I double-checked the files used in the file check, and the versions look good. Haven't dug much deeper; I see the code looks for some classes also, it might be failing there. A 2.0 (utf8) => 3.0 upgrade attempt has the same result. 
|
|
I attempted a 1.1 => 3.0 upgrade & got a WSOD with the following:
Pretty sure custom avatars didn't exist before 2.0? |
|
I attempted a 1.0 => 3.0 upgrade & got a different WSOD:
|
|
yabbse:
It's been a while since I've run these, always possible I'm grappling with environment issues. |
|
I took a peek at the 2.0 issue. First time seeing this code, so take this with a grain of salt... It finds all the files OK, Upgrade.php line 704. When it starts looking for the 2.0 classes is where there is an issue, Upgrade.php, line 718. It finds & loads the first one fine (SMF\Maintenance\Migration\v2_0\PostgreSQLFunctions), and loadClass() is properly returning true. So far, so good... But for some reason it still throws the exception on line 719. I believe it's falling back thru a closure & losing the "true" there. So, well hidden under the covers, it is throwing the error: Exception->__construct($message = 'SMF\Maintenance\Migration\v2_0\PostgreSQLFunctions does not exist') D:\wamp64\www\van2021\Sources\Maintenance\Tools\Upgrade.php:719 When in fact it was loaded OK. |
|
Note that loadClass actually does an include, which in turn, recursively includes MigrationBase & SubStepInterface immediately after PostgreSQLFunctions. It's possible there's an issue with one of those? Topic me, all appear to load ok. |
Sesquipedalian
force-pushed
the
3.0/upgrade_old_versions
branch
from
356c8c9 to
ddf782e
Compare
Just trying to clarify here. When you write "loadClass", do you mean "class_exists"? |
|
class_exists by default forces an autoload, which calls Composer's loadClass as well as many layers of other stuff... A fractal recursive maze of OOPsie. check_exists loads everything. If that's not desired, I think you can pass false as 2nd param. I'm not sure why that's failing though. Seems to be loading fine - including the layers of base classes & interfaces. Makes no sense. |
|
I think I found my new forum signature: "A fractal recursive maze of OOPsie." |
|
The last class loaded was SubStepInterface. This might make sense if there were an issue with SubStepInterface, but I don't see one. |
|
Gotcha. Hm, that's still odd. I don't have any ideas about it yet. I'll have to try again to see if I can reproduce the error locally. |
|
OK, some progress... My version of Composer was a few months old, 2.8.8. Latest version is 2.9.5, and the error was clearly deep in the heart of Composer, so... I did a composer self-update to 2.9.5, and now I'm getting further... But the error now is identical to the 1.1 error. And yes, I refreshed to get the most recent commits:
|
|
Odd the 2.1 => 3.0 upgrade was not impacted, but the 2.0 => 3.0 upgrade was... |
That should be an easy fix. I'll push a commit for that later tonight. |
|
Does that latest commit allow installation to proceed, @sbulen? |
|
Sorry, wasn't sure you wanted a retest yet. Reran 2.1 => 3.0 - OK Latin1:
utf8:
|
Sesquipedalian
force-pushed
the
3.0/upgrade_old_versions
branch
2 times, most recently
from
69a55d1 to
d92af01
Compare
|
I've now pushed up fixes for all issues reported so far. When you are able, @sbulen, please check out a new copy of this brach and try again. |
|
#9139 logged. We can loop back later. |
Sesquipedalian
force-pushed
the
3.0/upgrade_old_versions
branch
from
8d60c46 to
7936bca
Compare
<?php
/**
* Simple Machines Forum (SMF)
*
* @package SMF
* @author Simple Machines https://www.simplemachines.org
* @copyright 2024 Simple Machines and individual contributors
* @license https://www.simplemachines.org/about/smf/license.php BSD
*
* @version 3.0 Alpha 3
*/
declare(strict_types=1);
namespace SMF\Maintenance\Migration\v1_1;
use SMF\Config;
use SMF\Db\DatabaseApi as Db;
use SMF\Db\Schema;
use SMF\Maintenance\Maintenance;
use SMF\Maintenance\Migration\MigrationBase;
class PersonalMessages extends MigrationBase
{
/*******************
* Public properties
*******************/
/**
*
*/
public string $name = 'Updating personal message functionality';
/****************
* Public methods
****************/
/**
*
*/
public function execute(): bool
{
Maintenance::$tool->logProgress((string) __LINE__, true);
$existing_tables = Db::$db->list_tables();
Maintenance::$tool->logProgress((string) __LINE__, true);
// Rename the existing tables.
if (\in_array(Config::$db_prefix . 'instant_messages', $existing_tables)) {
Maintenance::$tool->logProgress((string) __LINE__, true);
$this->query(
'RENAME TABLE {db_prefix}instant_messages
TO {db_prefix}personal_messages',
);
Maintenance::$tool->logProgress((string) __LINE__, true);
}
if (\in_array(Config::$db_prefix . 'im_recipients', $existing_tables)) {
Maintenance::$tool->logProgress((string) __LINE__, true);
$this->query(
'RENAME TABLE {db_prefix}im_recipients
TO {db_prefix}pm_recipients',
);
Maintenance::$tool->logProgress((string) __LINE__, true);
}
// Update columns and indexes on pm_recipients table.
Maintenance::$tool->logProgress((string) __LINE__, true);
$table = new Schema\v1_1\PmRecipients();
Maintenance::$tool->logProgress((string) __LINE__, true);
$table->normalize();
// Data updates.
Maintenance::$tool->logProgress((string) __LINE__, true);
$this->query(
'UPDATE {db_prefix}pm_recipients
SET labels = {string:neg_one}
WHERE labels NOT RLIKE {string:regex} OR labels = {string:empty}',
[
'neg_one' => '-1',
'regex' => '[0-9,\-]',
'empty' => '',
],
);
// Rename columns in members table
Maintenance::$tool->logProgress((string) __LINE__, true);
$table = new Schema\v1_1\Members();
Maintenance::$tool->logProgress((string) __LINE__, true);
$existing_structure = $table->getCurrentStructure();
if (isset($existing_structure['columns']['im_ignore_list'])) {
Maintenance::$tool->logProgress((string) __LINE__, true);
$table->alterColumn($table->columns['pm_ignore_list'], 'im_ignore_list');
Maintenance::$tool->logProgress((string) __LINE__, true);
}
if (isset($existing_structure['columns']['im_email_notify'])) {
Maintenance::$tool->logProgress((string) __LINE__, true);
$table->alterColumn($table->columns['pm_email_notify'], 'im_email_notify');
Maintenance::$tool->logProgress((string) __LINE__, true);
}
Maintenance::$tool->logProgress((string) __LINE__, true);
$table->normalize();
Maintenance::$tool->logProgress((string) __LINE__, true);
$this->handleTimeout();
Maintenance::$tool->logProgress((string) __LINE__, true);
return true;
}
}That should dump a bunch of line numbers into the log file, so that we can at least figure out where things are going wrong. |
|
OK today's tests! Details box good - Thanks! 2.1 => 3.0 OK 2.0 latin1: 2.0 utf8: 1.1: LOG FROM 1.1: |
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Sesquipedalian
force-pushed
the
3.0/upgrade_old_versions
branch
from
7936bca to
156d344
Compare
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Sesquipedalian
force-pushed
the
3.0/upgrade_old_versions
branch
from
156d344 to
1dba385
Compare
Should be fixed in latest commits.
Can you try again with debug enabled so that we can see where the problem lies?
Should be fixed in latest commits.
Should be fixed in latest commits. |
|
Note I see this in the PHP log... Should have looked at this earlier... This includes errors from all yesterday's tests...
|
|
OK today's tests! Another timer nit... Timer looks better, but only increments after major steps are completed. This in the past was an indicator it was actually running, even with debug off. In fact, it said so on the screen... "The time elapsed increments from the server to show progress is being made!" That was helpful on large forum upgrades. I'm asking for debug on on all these, since that seems to provide better info. 2.1:
*** Nothing in the php/mysql/apache logs... This is the bottom of the upgrader log:
2.0 latin1:
PHP log: 2.0 utf8:
PHP log: 1.1:
PHP log: |
3 participants
I've had this sitting around nearly completed for months now, so I thought I'd take a few hours to finish it off and submit it.
This PR adds all the necessary table schemas, migration steps, and cleanup steps required to allow the upgrader to upgrade any version of SMF or even YaBB SE.
@jdarwood007, I know that we had previously discussed writing converters for old SMF versions rather than upgrader steps. However, when I sat down to do it, writing upgrader steps turned out to be much easier, because I could base them on the old upgrade scripts rather than trying to come up with entirely new logic. And since everything in the new SMF\Maintenance paradigm is so clearly defined and reliable, I don't think these new upgrader steps are going to be a burden to maintain the way the old upgrade scripts were.
@sbulen, is there any chance you could fire up that marvellous test suite of yours to make sure that these all work? Overall, I expect that it go well, but there are doubtless some copy-paste errors and typos hiding in there someplace.