Difference between revisions of "Rollover for the manual"

From Gramps
Jump to: navigation, search
m (Regenerate Version-Specific Content: For the Gramps user manual only the latest version should be listed on Category:Stub.)
 
(58 intermediate revisions by 4 users not shown)
Line 1: Line 1:
[[Category:Developers/General]]
+
{{man warn|Notes for the webmaster@gramps-project.org|Used in the maintenance of the Gramps website}}
On this page, 3.x refers to the 'old' (current) version of the manual - 3.z refers to the new, to be created, version...
+
Technical details on how the Gramps wiki [[User manual]] is updated for each release.
#Use phpmyadmin to backup all of the wiki tables!Locate all pages on the wiki that have 3.x in the title - from the wiki: toolbox, Special pages, All pages to get a full list of wiki pages. 
 
#Copy/paste it into text processor - it is in three columns separated by tabs.  Use regular expression search/replace to replace tabs with carriage returns.  Using GEDIT, best to replace \t with \n rather than \r.  You now have a list of all files on the wiki, one title per line.  Save it as a text file (I'll call it myfile.txt for this example).
 
#Use grep to find only the pages with 3.x in the title: ''grep "3\.x" myfile.txt > mynewfile.txt'' The backslash will make the full stop a literal rather than trying tobe part of a regular expression.
 
#Use a text editor to view the new file, and delete any pages that you don't want to roll over (That is, some pages may refer to 3.x in the title, but are not pages that we want duplicated into 3.z...)
 
#Copy/paste the new list of page titles into the wiki export using toolbox, Special pages, Export pages and create the xml to screen.  Be sure box for Include only the current revision, not the full history is ticked.
 
#Copy/paste the xml into a text processor and use search/replace to replace 3.x with 3.z
 
#Save the file (I'll call it export.xml for this example).
 
#Import the new pages into the wiki: toolbox, Special pages, Restricted special pages (available only to Sysop members of the wiki), Import pages.  Browse to find the .xml file and Upload file.
 
  
[NOTE: Updating through multiple xml files not necessary now, as we have command line access to our hosting account.] There are serious restrictions for both file size and processing time for the scripts - I had to split the .xml file into 8 pieces to get it all uploaded...
+
== Rolling over the Gramps user manual ==
 +
On this page, <code>5.x</code> refers to the 'old' (current) version of the manual - <code>5.z</code> refers to the new, to be created, version...
 +
====Backup wiki====
 +
Use phpmyadmin to backup all of the wiki tables (if phpmyadmin times out use the commandline )!
 +
* [https://au.godaddy.com/help/exporting-mysql-database-in-ssh-26367 Command line:] <code>mysqldump -p -h hostname -u username database > dbname.sql</code>
 +
====Process====
 +
#Locate all pages on the wiki that have <code>5.x</code> in the title - from the {{man label|wiki menu: [[Special:AllPages|Tools > Special pages > All pages]]}}, page by page copy the text to get a full list of wiki pages.
 +
#Copy/paste it into text processor - it may be in three columns separated by tabs.  If so, use regular expression search/replace to replace tabs with carriage returns.  Using GEDIT, best to replace <code>\t</code> with <code>\n</code> rather than <code>\r</code>.  You now have a list of all files on the wiki, one title per line.  Save it as a text file (I'll call it <code>myfile.txt</code> for this example).
 +
#Use grep to find only the pages with <code>5.x</code> in the title: ''<code>grep "5\.x" myfile.txt > mynewfile.txt</code>'' The backslash will make the full stop a literal rather than trying to be part of a regular expression.
 +
#Use a text editor to view the new file, and delete any pages that you don't want to roll over (That is, some pages may refer to <code>5.x</code> in the title, but are not pages that we want duplicated into <code>5.z</code>...)
 +
#Go to the {{man label|wiki menu: [[Special:Export|Tools > Special pages > Export pages]]}} and Copy/paste the new list of page titles into the {{man label|Add pages manually:}} section then {{man menu|Be sure the checkbox is ticked for {{man label|Include only the current revision, not the full history}}}} and then to create the xml to screen press the {{man label|Export}} button.
 +
#Copy/paste the xml into a text processor and use search/replace to replace <code>5.x</code> with <code>5.z</code>
 +
#Save the file (I'll call it <code>export.xml</code> for this example).
  
For the change to 3.4 and then from 3.4 to 4.0, I used a script in the maintenance directory to import xml from the command line.  The GoDaddy hosting service uses an old version of php from the command line, but I was successful with:
+
=== Import xml from the command line===
 +
For the change to <code>5.4</code> and then from <code>5.4</code> to <code>6.0</code>, I used the mediwiki script in the maintenance directory to import xml from the command line.  I was successful with (but I got an error in <code>importDump.php</code> and had to edit the file to ensure that <code>Maintenance.php</code> was included properly!):
  
'''-bash-3.2$ /usr/local/php5/bin/php importDump.php 34.xml'''
+
'''-bash-prompt$<code>php importDump.php export.xml</code>'''
  
 
followed by:
 
followed by:
  
'''-bash-3.2$ /usr/local/php5/bin/php rebuildrecentchanges.php'''
+
'''-bash-prompt$<code>php rebuildrecentchanges.php</code>'''
  
 
That should now have created all of the new pages.
 
That should now have created all of the new pages.
  
New pages created by this sort of import do not automatically get added into the search index of the wiki.  Use phpmyadmin from the cpanel and use the repair tool for the searchindex table.
+
===Update searchindex table===
 +
Before doing this backup the database.
  
Finally, if you want to set all the previous version pages to be protected, so only sysop mediawiki users are able to make changes (so all changes will be forced into the next set of pages...), you can use a series of sql statements in the form:
+
{{man warn|Update searchindex table|New pages created by this sort of import do not automatically get added into the search index of the wiki.''}}
  
UPDATE `grampsd_wiki`.`page` SET `page_restrictions` = 0x656469743d7379736f703a6d6f76653d7379736f70 WHERE `page`.`page_title` ='Title_of_the_page_in_the_wiki' collate utf8_unicode_ci LIMIT 1;
+
* Use phpmyadmin from the cpanel and use the repair tool(on the ''Operations'' tab and under ''table Maintenance>Repair table''for the <code>searchindex</code> table.
  
(You'll need to replace the spaces in the page title with underscores...)
+
===Set previous version pages to be protected===
  
I do that concatenation in a spreadsheet, creating the (long) SQL statements by combining a first part, the page names and the last part...
+
Finally, if you want to set all the previous older user manual version pages to be protected, so only sysop mediawiki users are able to make changes (so all changes will be forced into the next set of pages...), you can use a series of sql statements in the form:
 +
<pre>
 +
UPDATE `grampswiki`.`page` SET `page_restrictions` = 0x656469743d7379736f703a6d6f76653d7379736f70 WHERE `page`.`page_title` ='Title_of_the_page_in_the_wiki' collate utf8_unicode_ci LIMIT 1;
 +
</pre>
 +
({{man menu|You'll need to replace the spaces in the page title with underscores...}})
 +
 
 +
{{man note|Beginning with MediaWiki 1.10|Page protection controls were moved to the page restrictions table, so this field will be empty in databases generated by more current versions of MediaWiki. However, this field is still used in current versions of MediaWiki for rows generated by older versions of MediaWiki!
 +
<br>
 +
https://www.mediawiki.org/wiki/Manual:Page_restrictions_table
 +
<br>
 +
https://www.mediawiki.org/wiki/Manual:Page_table#page_restrictions }}
 +
 
 +
Do that by using concatenation in a spreadsheet (call it <code>protectoldpages.xls</code>), creating the (long) SQL statements by combining a first part, the page names and the last part.  You'll need to search for any single quotes and put a backslash in front of them in the page names first, too...
  
 
That updates a blob field to force protection without having to do the pages individually.  You need to do the above sql statement for each page that you want to protect...
 
That updates a blob field to force protection without having to do the pages individually.  You need to do the above sql statement for each page that you want to protect...
  
*Screenshots
+
* Copy the resulting SQL statement to a text file <code>protectoldpages.txt</code>
As there is no [[Available_screenshots|naming scheme]] we are trying to follow a rule for manual updgrade :
+
* check the SQL statements syntax [https://www.piliapp.com/mysql-syntax-check/ here]
filename-{number of version}-{lang}.extension
+
* Then run the SQL statement by either using phpmyadmin("SQL" tab to run the query) or the commandline.
 +
 
 +
Backup the database.
 +
 
 +
===Screenshots===
 +
As there is no [[Screenshots|naming scheme]] we are trying to follow a rule for manual upgrade :
 +
* <code>filename-{number of version}-{lang}.extension</code>
 
ex:
 
ex:
Edit-person-31-en.png
+
* <code>Edit-person-50-en.png</code>
 
We need to know if we keep the {number of version} or if we use the new one on migration (3.1->3.2->4.0->15.0...) ?
 
We need to know if we keep the {number of version} or if we use the new one on migration (3.1->3.2->4.0->15.0...) ?
 +
 +
===Regenerate Version-Specific Content===
 +
Update files dependency list :
 +
''The documentation assumes the default (English-US) language on Linux''
 +
 +
Edit the manual section :
 +
 +
<code> https://gramps-project.org/wiki/index.php?title=Gramps_{{Version manual}}_Wiki_Manual_-_Command_Line&action=edit&section=8</code>
 +
 +
 +
Copy (or pipe) the updated Gramps installation output from the command line Version option :
 +
 +
<code> gramps -v</code>
 +
 +
Replace any user-specific directory information with '''<~username>'''
 +
 +
If appropriate, please reference the Gramps "[[Gramps_Glossary#user_directory|User Directory]]" Glossary term entry instead of redundantly explaining the directory paths under any of the multiuser OSes. This glossary entry include a drill-down link to the Appendix describing how to find the directory for a particular fork of Gramps
 +
 +
<code>&#91;&#91;Gramps_Glossary#user_directory|User Directory]]</code>
 +
 +
Gramps user directory available on various operating systems.
 +
 +
===Remove "STUB" template from the previous User manual===
 +
For the Gramps user manual only the latest version should be listed on [[:Category:Stub]].
 +
 +
==Version manual template==
 +
Update the current ( {{Version manual}} ) version number using on [[:Template:Version_manual]] for pages outside of the user manual.
 +
 +
Manually update any REDIRECT pages also eg: "[https://gramps-project.org/wiki/index.php?title=Third-party_Addons&action=edit Third-party Addons]" link redirects to 5.1 Addons etc..
 +
 +
==Email announcement==
 +
Send the following email announcement (see embedded email) to the mailing list.
 +
<!--
 +
Hello,
 +
 +
Due to scheduled maintenance the Gramps Wiki services will be unavailable for most of the day on Sunday, 23rd of June 2019.
 +
 +
Please Note: during this offline period you may still be able to view the wiki.
 +
 +
The wiki software Mediawiki is being upgraded.
 +
 +
Then Gramps User Manual is being rolled over for the upcoming release of Gramps 5.1.x.
 +
 +
Apologies for any inconvenience that this may cause and thank you for patience and understanding.
 +
 +
Regards,
 +
 +
Sam
 +
Volunteer Website Administrator
 +
The Gramps Project
 +
 +
-->
 +
<!--
 +
Hello,
 +
 +
The rollover of the wiki user manual pages in preparation for the 5.1 version has been completed.
 +
 +
Please let me know if you see any obvious problems with the rollover.
 +
 +
Over the next few days, I'll be making all of the 5.0 manual pages non-editable, so please only edit the new 5.1 manual.
 +
 +
Sam
 +
Volunteer Website Administrator
 +
The Gramps Project
 +
 +
-->
 +
==External Site changes==
 +
* Some of the [https://discourse.gramps-project.org Discourse forum] '''automatic linking of key phrases''' point to sections in the current manual.  A user with "Leader" or higher trust setting should review and update the Admin -> Customize -> [https://gramps.discourse.group/admin/customize/watched_words/action/link Watched Words -> Link] list. This is most easily accomplished by:
 +
# using the discourse Download button to acquire a CSV of the list (which is an unqualified comma delimited list with a "phrase,url" on each line)
 +
# edit it in a text editor
 +
# replace the versions in the list
 +
# save the file
 +
# use the "Clear All" discourse button
 +
# use the "Add from file" discourse button to upload the modified list
 +
# refresh the Discourse page manually as that seems to be quirky
 +
# use the "Test" discourse button to verify a couple linkify phrases
 +
 +
==See also==
 +
* Generating the [[Manual_Generation_5.1|Offline User Manual]] (includes template scripts for English & 3 translations)
 +
* [[User manual]]
 +
* [[Manual Generation]] - <small>revised 28 Aug 2023 for Gramps 5.1.6</small>
 +
* [[Manual Generation 3.0]]
 +
* [[What to do for a release]]
 +
 +
[[Category:Developers/General]]

Latest revision as of 21:27, 29 December 2023

Gnome-important.png
Notes for the [email protected]

Used in the maintenance of the Gramps website

Technical details on how the Gramps wiki User manual is updated for each release.

Rolling over the Gramps user manual

On this page, 5.x refers to the 'old' (current) version of the manual - 5.z refers to the new, to be created, version...

Backup wiki

Use phpmyadmin to backup all of the wiki tables (if phpmyadmin times out use the commandline )!

  • Command line: mysqldump -p -h hostname -u username database > dbname.sql

Process

  1. Locate all pages on the wiki that have 5.x in the title - from the wiki menu: Tools > Special pages > All pages, page by page copy the text to get a full list of wiki pages.
  2. Copy/paste it into text processor - it may be in three columns separated by tabs. If so, use regular expression search/replace to replace tabs with carriage returns. Using GEDIT, best to replace \t with \n rather than \r. You now have a list of all files on the wiki, one title per line. Save it as a text file (I'll call it myfile.txt for this example).
  3. Use grep to find only the pages with 5.x in the title: grep "5\.x" myfile.txt > mynewfile.txt The backslash will make the full stop a literal rather than trying to be part of a regular expression.
  4. Use a text editor to view the new file, and delete any pages that you don't want to roll over (That is, some pages may refer to 5.x in the title, but are not pages that we want duplicated into 5.z...)
  5. Go to the wiki menu: Tools > Special pages > Export pages and Copy/paste the new list of page titles into the Add pages manually: section then Be sure the checkbox is ticked for Include only the current revision, not the full history and then to create the xml to screen press the Export button.
  6. Copy/paste the xml into a text processor and use search/replace to replace 5.x with 5.z
  7. Save the file (I'll call it export.xml for this example).

Import xml from the command line

For the change to 5.4 and then from 5.4 to 6.0, I used the mediwiki script in the maintenance directory to import xml from the command line. I was successful with (but I got an error in importDump.php and had to edit the file to ensure that Maintenance.php was included properly!):

-bash-prompt$php importDump.php export.xml

followed by:

-bash-prompt$php rebuildrecentchanges.php

That should now have created all of the new pages.

Update searchindex table

Before doing this backup the database.

Gnome-important.png
Update searchindex table

New pages created by this sort of import do not automatically get added into the search index of the wiki.

  • Use phpmyadmin from the cpanel and use the repair tool(on the Operations tab and under table Maintenance>Repair tablefor the searchindex table.

Set previous version pages to be protected

Finally, if you want to set all the previous older user manual version pages to be protected, so only sysop mediawiki users are able to make changes (so all changes will be forced into the next set of pages...), you can use a series of sql statements in the form:

UPDATE `grampswiki`.`page` SET `page_restrictions` = 0x656469743d7379736f703a6d6f76653d7379736f70 WHERE `page`.`page_title` ='Title_of_the_page_in_the_wiki' collate utf8_unicode_ci LIMIT 1;

(You'll need to replace the spaces in the page title with underscores...)

Gramps-notes.png
Beginning with MediaWiki 1.10

Page protection controls were moved to the page restrictions table, so this field will be empty in databases generated by more current versions of MediaWiki. However, this field is still used in current versions of MediaWiki for rows generated by older versions of MediaWiki!
https://www.mediawiki.org/wiki/Manual:Page_restrictions_table
https://www.mediawiki.org/wiki/Manual:Page_table#page_restrictions

Do that by using concatenation in a spreadsheet (call it protectoldpages.xls), creating the (long) SQL statements by combining a first part, the page names and the last part. You'll need to search for any single quotes and put a backslash in front of them in the page names first, too...

That updates a blob field to force protection without having to do the pages individually. You need to do the above sql statement for each page that you want to protect...

  • Copy the resulting SQL statement to a text file protectoldpages.txt
  • check the SQL statements syntax here
  • Then run the SQL statement by either using phpmyadmin("SQL" tab to run the query) or the commandline.

Backup the database.

Screenshots

As there is no naming scheme we are trying to follow a rule for manual upgrade :

  • filename-{number of version}-{lang}.extension

ex:

  • Edit-person-50-en.png

We need to know if we keep the {number of version} or if we use the new one on migration (3.1->3.2->4.0->15.0...) ?

Regenerate Version-Specific Content

Update files dependency list : The documentation assumes the default (English-US) language on Linux

Edit the manual section :

https://gramps-project.org/wiki/index.php?title=Gramps_5.2_Wiki_Manual_-_Command_Line&action=edit&section=8


Copy (or pipe) the updated Gramps installation output from the command line Version option :

gramps -v

Replace any user-specific directory information with <~username>

If appropriate, please reference the Gramps "User Directory" Glossary term entry instead of redundantly explaining the directory paths under any of the multiuser OSes. This glossary entry include a drill-down link to the Appendix describing how to find the directory for a particular fork of Gramps

[[Gramps_Glossary#user_directory|User Directory]]

Gramps user directory available on various operating systems.

Remove "STUB" template from the previous User manual

For the Gramps user manual only the latest version should be listed on Category:Stub.

Version manual template

Update the current ( 5.2 ) version number using on Template:Version_manual for pages outside of the user manual.

Manually update any REDIRECT pages also eg: "Third-party Addons" link redirects to 5.1 Addons etc..

Email announcement

Send the following email announcement (see embedded email) to the mailing list.

External Site changes

  • Some of the Discourse forum automatic linking of key phrases point to sections in the current manual. A user with "Leader" or higher trust setting should review and update the Admin -> Customize -> Watched Words -> Link list. This is most easily accomplished by:
  1. using the discourse Download button to acquire a CSV of the list (which is an unqualified comma delimited list with a "phrase,url" on each line)
  2. edit it in a text editor
  3. replace the versions in the list
  4. save the file
  5. use the "Clear All" discourse button
  6. use the "Add from file" discourse button to upload the modified list
  7. refresh the Discourse page manually as that seems to be quirky
  8. use the "Test" discourse button to verify a couple linkify phrases

See also