WBCE CMS Forum

WBCE CMS – Way Better Content Editing.

Du bist nicht angemeldet.

#1 23.01.2023 22:21:26

losttrip
Mitglied

Import old News Posts to NWI

I am migrating a WB 2.10 site to WBCE 1.5.4.  Actually I am recreating the site from scratch in WBCE 1.5.4.

I knew it would be tedious, but worth the effort, in my opinion.  But I have hit a serious bottleneck.  The old site has hundreds of News posts.  I started to add each post on the new site, but it's taking much longer than I imagined.

I looked at the databases for the two sites (old) "mod_news_posts" and (new) "mod_news_img_posts", and I am pretty confident I could export the old database, and edit it in a text editor to work in the new database.  Although I don't have a lot of experience with this.

old News Database

new NWI database

But I have a feeling that's not all there is to it.  For instance, I see php files for each post under "pages/posts"
e.g.,

[== PHP ==]
<?php
$page_id = 2;
$section_id = 3;
$post_id = 4;

define("POST_SECTION", $section_id);
define("POST_ID", $post_id);
require("../../config.php");
require(WB_PATH."/index.php");
?>

Would I need to manually create each of these files as well?

Is there anything else I need to check or look out for?

Thank you for your time.

Offline

#2 23.01.2023 22:33:19

florian
Administrator

Re: Import old News Posts to NWI

NWI has an import tool for news posts. So you can try the following:
1) Create a database dump from the old news_posts table
2) Install the old news module on the new WBCE  page (https://addons.wbce.org/pages/addons.php?do=item&item=7)
3) Open the dump of the old News_img table with an editor like Notepad++ and change the first two values after the opening bracket according to the page id / section id of the old news module in your new site
4) import the changed sql file into the database of your new WBCE site (change prefix of table to import if necessary)
5) Check (in the backend)  if the news posts appear on the news page
6) Create a new, empty NWI page
7) There sholud appear an "import from" select box, choose the news section you just filled
8) If the import was successful, delete the news page from which you imported the posts
9) Uninstall the news module

Beitrag geändert von florian (24.01.2023 06:31:30)


Code allein macht nicht glücklich. Jetzt spenden!

Offline

#3 24.01.2023 04:02:35

losttrip
Mitglied

Re: Import old News Posts to NWI

Thank you.  I will try this.

I did do a few forum searches (English) but wasn't successful in finding this information.

Offline

#4 25.01.2023 23:03:08

losttrip
Mitglied

Re: Import old News Posts to NWI

Hi Florian.  Using your instructions, I was able to import all of the old News posts to NWI.  The posts all appear in the backend and the frontend.

But when you click "read more" on any post, you go to a white page.  I checked the file system and no php files exist for any of the post under "pages/posts".  I opened one of the NWI posts in the backend and saved it and the php file was created for that post.  Is a re-save of EVERY post necessary to complete this migration?

Offline

#5 26.01.2023 08:28:18

florian
Administrator

Re: Import old News Posts to NWI

Hm, that ist strange. Actually the access files should have been recreated on import and no further action should be necessary.

Anyway, I've created (copied&pasted) a quick&dirty script to create all accessfiles of a given section.

1) Add a code2 section to the NWI page with the missing access files.
2.) Paste the following code into it:

global $database;
$posts = array();
$nwisection = 39;

$query = $database->query("SELECT * FROM `".TABLE_PREFIX."mod_news_img_posts` WHERE `section_id`=".$nwisection);
    if ($query->numRows() > 0) {
        while ($post = $query->fetchRow()) {
            $posts[] = $post;
        }
    }

include_once WB_PATH.'/modules/news_img/functions.inc.php';

for ($i=0; $i<sizeof($posts); $i++) {
 

$old_link = $posts[$i]['link'];

// potential new link
$post_link = '/posts/'.page_filename($posts[$i]['title']);
// make sure to have the post_id as suffix; this will make the link unique (hopefully...)
if (substr_compare($post_link, $posts[$i]['post_id'], -(strlen($posts[$i]['post_id'])), strlen($posts[$i]['post_id']))!=0) {
    $post_link .= PAGE_SPACER.$posts[$i]['post_id'];
}

// Make sure the post link is set and exists
// Make news post access files dir
make_dir(WB_PATH.PAGES_DIRECTORY.'/posts/');
$file_create_time = '';
if (!is_writable(WB_PATH.PAGES_DIRECTORY.'/posts/')) {
    $admin->print_error($MESSAGE['PAGES_CANNOT_CREATE_ACCESS_FILE']);
} else {
    // We need to create a new file
    // First, delete old file if it exists
    if (file_exists(WB_PATH.PAGES_DIRECTORY.$old_link.PAGE_EXTENSION)) {
        $file_create_time = filemtime(WB_PATH.PAGES_DIRECTORY.$old_link.PAGE_EXTENSION);
        unlink(WB_PATH.PAGES_DIRECTORY.$old_link.PAGE_EXTENSION);
    }
    if ($page_id != $old_page_id or $section_id != $old_section_id) {
        $file_create_time = '';
    }
    // Specify the filename
    $filename = WB_PATH.PAGES_DIRECTORY.'/'.$post_link.PAGE_EXTENSION;
    mod_nwi_create_file($filename, $file_create_time, $posts[$i]['post_id'],$nwisection,$page_id);
    echo "created ".$posts[$i]['link'].'<br>';
}
  
}

3.) Change the value for $nwisection according to the actual nwi section.
attachment.php?item=3335&download=1

4.) Click on the "save" button of the code2 section.
5.) Call the frontend view of the page. Below the news items a list like "created /posts/some-text-here1" and so on appears.
6.) Check some news posts in the frontend.
7.) Remove the code 2 section.

To make sure that really all access files exist, install the System information tool from AOR. On the "pages" tab, below the regular pages, you can check if all NWI access files exist and create missing ones if necessary.

Beitrag geändert von florian (26.01.2023 08:31:56)


Code allein macht nicht glücklich. Jetzt spenden!

Offline

Liked by:

losttrip

#6 26.01.2023 23:09:33

losttrip
Mitglied

Re: Import old News Posts to NWI

florian schrieb:

Hm, that ist strange. Actually the access files should have been recreated on import and no further action should be necessary.

I will definitely give your script a try.  Thank you for going to that trouble.

But, I would be interested in determining what went wrong to cause the access files to not be created on import.  Just in case I did something wrong, or missed something... and so that I understand the process for future reference.

Is it possible that the access files were not "recreated" because there were no access files for the original posts that were imported from the News module page?  There were no access files for those News posts because I did a database table import of the edited sql file.

Or does the NWI "import" function not look at the original News page access files in order to create the new NWI access files on import?

Offline

#7 27.01.2023 01:08:24

florian
Administrator

Re: Import old News Posts to NWI

I don't know what went wrong. The NWI import tool should create the access files on import in any case, wether they exist or not, since they contain other values than the original files. Also the /posts directory should be created if does not exist and if the directory isn't writeable, there should appear an error message.
You can check if there are any corresponding entries in the error log which might explain the missing access files.
Also I'm interested if the script worked.


Code allein macht nicht glücklich. Jetzt spenden!

Offline

#8 27.01.2023 03:00:26

losttrip
Mitglied

Re: Import old News Posts to NWI

Hmmm.... I am torn.  The troubleshooter in me wants to go back and start over and do your original steps again, just to see if I get the same result.  But, the practical side says just continue...

I already checked the error log and there was nothing there.

I installed the  System information tool and verified that all news posts were marked "missing".

I ran your script on both of my new NWI pages.  It worked as far as I can tell.  I checked  System information tool again and all posts (hundreds)  are marked "exists" except three still "missing".

I noted two of the "missing" post titles have characters in them (slash and exclamation points), but the third looked okay, until I realized there is a space at the end of the title.

I assumed you must have had checks in your script which caused post titles like this to fail creating access files.  But then I checked my file system and there are access files created, but their names don't match the post title, which I guess is what is causing them to be marked "missing" and for the "read more" links to still go to a blank page.

I'm only providing all this extra info in case it might be helpful to you, in case you want to further develop your script.  I can fix things up for these three posts, for my needs.

I checked the file system and the corresponding access file titles are:

missing 1
/posts/congratulations-157    ->    Access File  ->  congratulations----157.php  (extra three hyphens in place of the exclamation marks)

missing 2
/posts/next-meetingtraining-night-49    ->    Access File  ->    next-meeting-training-night-49.php  (extra hyphen in place of dash)

missing 3
/posts/pics-from-the-2016-kefa-golf-tournament-100    ->    Access File  ->      pics-from-the-2016-kefa-golf-tournament--100.php  (note the extra hyphen before the post id number, which is obviously caused by the blank space at the end of the post title)



FYI - I did get a bunch of this error in the log after running the script:

2023-01-27T01:13:45+00:00 [Warning] /temp/modules/code2/section_26.php.inc:[42] from /framework/frontend.functions.php:[391] "Undefined variable $old_page_id" 

Thank you again for your help!  You've saved me hours of time.

Edit - I just clicked "Recreate" on all three of the problem posts, and the correctly titled access files were created and the news post pages now work.

Thanks again!

Beitrag geändert von losttrip (27.01.2023 03:07:50)

Offline

#9 27.01.2023 03:29:29

losttrip
Mitglied

Re: Import old News Posts to NWI

Next issue I have encountered.

All of the links in the NWI posts (which were imported from old News section) to files in the media folder appear like this in the source code view and lead to 404 pages.

 <a href="{SYSVAR:MEDIA_REL}/subfolder/example_file.pdf">here</a>

Offline

#10 27.01.2023 08:24:23

florian
Administrator

Re: Import old News Posts to NWI

I assumed you must have had checks in your script which caused post titles like this to fail creating access files.

Actually I encountered that issue too, that for some reasons some of the access files of my test installation were not created. I did not figure out why and it was not intended.
I must confess that I mainly copied & pasted the script code from the save_post.php and just put the array and the loop around to execute the creation script for all news posts.
I have also no idea why it works with the System Information tool but not in the helper script.

All of the links in the NWI posts (which were imported from old News section) to files in the media folder appear like this in the source code view and lead to 404 pages.

About the SYSVAR:MEDIA_REL issue

Oops. I thought, WBCE would replace that placeholder globally, but points out: nah, this has to be done by each module of its own.
But there's a simple workaround too.
1.) Download the zip attached to this post.
2.) Go to Admintools > Output Filter Dashboard.
3.) Switch to "Advanced options" (Change the value of "Show Advanced Options" to enabled and click on save), if necessary.
4.) Click on upload filter (plugin).
5.) Select the previously saved zip file and click on Upload.

A new filter named "media_rel_to_path" appears, it should be activated already. This filter is executed during the rendering of all pages and posts in the frontend and will replace all left over appearances of {SYSVAR:MEDIA_REL} with the actual link to the media folder.

Edit / addition: just learned that it's not necessary to run that filter on every single page.
So after you installed the filter, click on it and change the filter type from "Page" to "Module (last)" and apply it to "various > News with images".
attachment.php?item=3338&download=1

Beitrag geändert von florian (27.01.2023 12:41:37)


Code allein macht nicht glücklich. Jetzt spenden!

Offline

Liked by:

stefanek, losttrip

#11 27.01.2023 16:28:03

losttrip
Mitglied

Re: Import old News Posts to NWI

I feel bad even saying this, since you spent the time providing this output filter and these thorough instructions, but could I alternately do a search for

<a href=\"{SYSVAR:MEDIA_REL}/

and replace with the actual link to the media folder

<a href=\"https://www.yourwebsite.com/media/

thereby avoiding the need for this output filter?

While I understand the concept of these output filters, I don't know anything about their impact on page loads, or whether I may run into issues with their implementation in the future.  My feeling is I would rather not use an ongoing workaround, if I can manually make the global correction once and not have to rely on an ongoing workaround.  But, like I said, I am grateful that you provided a working solution, and I am just looking for your informed opinion.

----------

Just trying to help with the following friendly suggestion:
Perhaps, for future reference to others who might find themselves in the same situation I was in, and find this thread, we could flesh out a few more of the manual checks from Step 3 in your initial instructions about searching and replacing in the old News "mod_news_posts" database dump).

My attempt at contributing to your initial help steps:

3) Open the dump of the old News_img table with an editor like Notepad++ and change the first two values after the opening bracket according to the page id / section id of the old news module in your new site

a) section id / page id
For me, it was the second and third values ('section_id' and `page_id`) that needed to be changed, as the first value was the "post_id" (see 'mod_news_posts' table structure and some examples of post data, below).  Since I had multiple News pages on my site, I needed to make sure I did the corresponding search and replace for each.

-- Table structure for table `mod_news_posts`
--

CREATE TABLE `mod_news_posts` (
  `post_id` int(11) NOT NULL,
  `section_id` int(11) NOT NULL DEFAULT 0,
  `page_id` int(11) NOT NULL DEFAULT 0,
  `group_id` int(11) NOT NULL DEFAULT 0,
  `active` int(11) NOT NULL DEFAULT 0,
  `position` int(11) NOT NULL DEFAULT 0,
  `title` varchar(255) NOT NULL DEFAULT '',

---------
Examples:

(4, 7, 3, 0, 1, 1, 'November 2008 Meeting...
(5, 7, 3, 0, 1, 2, 'December 2008 Meeting...

(170, 50, 18, 8, 1, 66, 'Meeting...
(171, 50, 18, 8, 1, 67, 'May Minutes...

Additional values worth checking for:

b) wblink
I had to do a search for all the different instances of "wblink" in my database and replace the link target page ID from the original site with the link target page ID of the new site. ( e.g., original site [wblink29] -> new site [wblink5] )

c) SYSVAR:MEDIA_REL
If you feel it would be helpful and suitable, this could outline the search and replace of this placeholder.

Search for

<a href=\"{SYSVAR:MEDIA_REL}/

replace with:

<a href=\"https://www.yourwebsite.com/media/

I hope this helps.  Thank you once again!

Beitrag geändert von losttrip (27.01.2023 16:29:47)

Offline

Liked by:

florian

#12 27.01.2023 16:37:46

stefanek
Developer

Re: Import old News Posts to NWI

Hello losttrip,

there is an updated version of the NWI module (Florian updated it 2 hours ago).
Your best bet would be to change the files with the new ones that were changed:
https://github.com/WBCE/News-with-Image … 4145dc5a1d

I wouldn't change the SYSVAR to hardcoded domain url if I wouldn't need to.

The OpF Solution. It doesn't pose any critical danger but you are best served to just replace the files of the NWI update.

Best regards,
Christian


“Success is the progressive realization of a worthy ideal.” ― Earl Nightingale

Offline

#13 27.01.2023 18:41:55

losttrip
Mitglied

Re: Import old News Posts to NWI

Understood.  Thanks Christian and Florian.

Offline

Liked by:

stefanek, florian

#14 27.01.2023 23:42:41

mrbaseman
Developer

Re: Import old News Posts to NWI

I haven't  read the whole thread in detail but scanned roughly through the whole story. Sorry, I didn't find the time to follow the forum regularly in the past weeks.

I have initially contributed the import feature. For some reason which I don't remember the path where news with images stores the images, and perhaps the access files, too has changed later on. The import part wasn't adapted anymore, since most wbce installations had already switched from topics module or news module to news with images (and the import script is really a huge piece of code which usually is called only once. So it's a lot of maintenance work for a feature rarely used. It also never was intended as a fully complete solution, but rather an integrated helper to do most of the work when you need to convert a huge number of posts)

The SYSVAR:MEDIA_REL in the news module is something which was introduced by the WB colleagues later on in their subsequent versons of the news module.

The [wblink] is a problem when you import a table from a different installation, where the page ids are different.

Offline

Liked by:

stefanek

Fußzeile des Forums

up