# MySword for Android Migration Tools v.3.1

### May 19, 2015

How to use these set of Migration tools.

Note that the executable files were designed to work in the Console (Command prompt, or what many people know as the DOS prompt).

1. What's New in Version 3.1
2. What's New in Version 3.0
3. What's New in Version 2.4
4. What's New in Version 2.3
5. What's New in Version 2.2
6. What's New in Version 2.1
7. What's New in Version 2.0
8. What's New in Version 1.2
9. What's New in Version 1.1
10. Installation
11. Migrating TheWord Bible Modules
12. Migrating TheWord Commentary Modules
13. Migrating TheWord Dictionary Modules
14. Migrating TheWord Book/Devotional/Graphics Modules
15. Migration Errors
16. Known Issues

1. ### What's New in Version 3.1

   1. Support for books with content type jpg and png like for some maps converted from e-Sword to The Word.
   2. RTF2MySwordHTML.dll RTF and RVF migration of links improvements for Book, Dictionary and Commentary migration tools:
      1. Links to other modules with id or title in links are now fully supported. For modules with id given by TheWord so that the file name is not in the link, TheWordIdMap.txt must be created to map the name like d10 or c4 to the actual file that must exist (in each line, the id must be followed by a tab character then the mapped file name).
      2. Links to modules with non-English/Latin names and links with title that are non-English/Latin are now supported.
      3. Self-links with titles that are non-English are now supported.
      4. Bible references to specific translation is now supported. The TheWordIdMap.txt can also be used to map a case of file name (and not abbreviation because The Word seems to favor Latin file names if the abbreviation is non Latin) to abbreviation for MySword use.
   3. Other RTF2MySwordHTML.dll RTF conversion improvements:
      1. \caps for capitalization command is now supported (it is used by some old modules for list subsequent items' first letter of the first word).
      2. Underlined Book_#:# Bible references are now directly supported in Use Open Office (it is used by some old modules) without detecting the verses in The Word then converting back to RTF.
      3. \line is now converted to HTML break
   4. RTF to HTML conversion via OpenOffice improvements:
      1. Support for Libre Office 5 which used small letters in HTML tags, font size enclosed in double quotes, and does not URL encode the pipe (|) used in links making them not work properly because of the |_AUTODETECT_| not removed. NOTE: Libre Office is unreliable in migrating RTF content when it gets larger. In case the records you are migrating are large like in Barnes, it is advisable to reduce RTF2HTMLOO_GroupCount so that some text in one record will not mixed up with the next. It is even more advisable to use Open Office instead.
      2. New setting default to false: RTF2HTMLOO_CleanEmptyParagraphs. This is useful for Poole's commentary from TheWord which has empty paragraphs which embed HTML break (BR tag) rendering them as extra line space but not very good for mobile device. The non-OpenOffice migrator migrates also empty paragraphs but without embedding breaks that is why they are ignored and not rendered.
      3. New lines (CRLF) are now converted to space so that quoted/phrase search will properly work.
      4. In case saving the temporary RTF to rtf.rtf failed due to access denied because previous group's process is still using it, rtf1.rtf will be used instead so conversion can proceed.
   5. Commentary migration bug fix: When using Open Office option, some self links are not corrected (points to dictionary) because the display text contained new lines.

2. ### What's New in Version 3.0

   1. Dictionary migrator improvements:
      1. Added migration of relative order (use MySword 5.7).
      2. Duplicate entries are no longer added as a separate entry with added number, instead they are appended to the end of the previous similar entry separated by a horizontal rule.
   2. Dictionary/Book migrator improvements:
      1. Links with bookmark (anchor) are now properly migrated which are useful for popups. E.g. LSJ (try creating bookmarks and hyperlinks to these bookmarks within the same entry and from other entries. Note that only MySword 6.2 supports bookmark range, older versions only support no range [Until (passage) is set to None]. The options for Until (passage) combo box:
         1. [None] - Dictionary behaves like [Next] but Journal/Book like [None] showing whole content but scrolling to the bookmark
         2. [Next] - means text from the bookmark up to before the next bookmark (if any) only will be included
         3. bookmark - means up to before this bookmark will be included
   3. RTF2MySwordHTML.dll RTF conversion improvements:
      1. Links within links and some nested UNICODE are now properly migrated by performing 2 passes for links and 3 passes for other tags.
      2. http links are now properly migrated.
      3. Links of the following form are now properly migrated:

             {\field{\*\fldinst{HYPERLINK "..."}}{\fldrslt ...}}

         Prior, \fldinst and HYPERLINK should only be separated by a space.
      4. Fixed bug: LSJ Greek spaces between words being removed.
      5. Fixed bug: RTF is not converted when the Module's content is RVF but the entry is RTF.
      6. Fixed bug: popup links with text containing \'XX where X is a hexadecimal digit are not converted.
      7. Proper support for UTF-8 RTFEncodingContent for RTF \'XX as UTF-8 requires encoding by block and not by byte.
   4. New Experimental RTF to HTML conversion via OpenOffice (required to be installed) which can include formatting not supported by RVF converter nor plain RTF converter like colors (foreground and background), margins, alignment (center and right), tables, etc. Note that the conversion is slower compared to the simpler RTF converter; for example, Eastons dictionary with 3965 entries converts in 1.5 minutes while it is just 15 seconds in the simple RTF converter.

      The following are the config's appSettings entries to use the feature:

      1. RTF2HTMLViaOpenOffice - must be set to true to enable the feature
      2. RTF2HTMLOO_GroupCount - the number of records to batch convert through one call to OpenOffice for speed. If the setting is 1, every record will mean one call while 100 will mean 100 records to aggregate to a single call. Eastons dictionary with 3965 records converted using group count of 1 required 16 minutes (very slow) while a setting of 100 to 200 required only 1.5 minutes. Note that if the setting is greater than 1, all records must be in RTF, otherwise an error will be thrown.

         Limitations: records with different RTF color tables will have problems when grouped that is why the group count will be smaller as only the same consecutive records with the same color tables will be grouped. In case all records have unique color tables or alternating different color tables, the effect will be a group count of 1. RTF has built-in standard palette of 16 colors (intense and mid-intensity red, green, blue, cyan, magenta, yellow, white and black). If all records use the standard palette then they will all have the same color table.

      3. RTF2HTMLOO_ReformatLists - must be set to true if the emulated list as indented paragraphs should be converted into HTML bullet and number lists.
      4. RTF2HTMLOO_CleanFont - must be set to true if HTML font tags should be removed. Note that it is advisable to remove them as they will affect the text scaling in MySword.
      5. RTF2HTMLOO_CleanTopBottomMargins - must be set to true if the style attribute entries for margin-top, margin-bottom and line-height should be removed.
      6. RTF2HTMLOO_ThemeColor - must be set to true if the embedded colors should be made compatible with MySword themes light or dark background.
      7. RTF2HTMLOO_MinFontSize - set to 10 if you want 10 pt and below, as normal
      8. RTF2HTMLOO_MaxRetrySaveHTML - Maximum number of tries to retry save as HTML as OpenOffice may fail the first time. 3 is advisable.

      Note that if there are records that contained non-English/Latin characters like Greek and Hebrew and they do not have the \rtf header and correct \fonttbl, garbage characters will result on those text (e.g. some records in VWS like Mark 1:1). These records will have to be corrected in TheWord by converting the module first to RVF (make it a first User editable module) via Detect verses, then convert it back to RTF.

3. ### What's New in Version 2.4

   1. Bible migrator improvements:
      1. biblebase.sqlite not in the folder where the Bible is will no longer throw an exception.
      2. Simple support for verse.rule(s) requiring MySword 5.1. Please note that toggle and TheWord links are not supported so they are dropped. Also, you may need to replace some rules to be more theme-independent. e.g.

             verse.rule="<HE>(.+?)<He>" "<font name=[Hebrew] size=+2 color=#800000>$1</font>"

         Can be replaced by:

             verse.rule="<HE>(.+?)<He>" "<span class='red' style='font-size:1.6em'>$1</span>"

         Or:

             verse.rule="<HE>(.+?)<He>" "<span class='red'>$1</span>"

      3. Bibles with version.date that is only has year will no longer throw an exception.
      4. Included the following fields in the migration if present (displayed in MySword About Module): title (set to description if not present), publisher, author, creator, source, language and editorial comments.
      5. description with multiple lines of text is now possible (prior, only the first line is read).
      6. SQLite database compaction (VACUUM command) is done to make the file size a bit smaller but ensuring the Details record is still found at the end of the file (used as flag for complete download).
      7. The migrator will try to convert embedded font color to MySword standard: red, orange, brown, yellowgreen, green, bluegreen, blue, violet, pink and gray. Almost black color will be removed so that the normal text color will be used.
   2. Migrator improvements:
      1. It is now possible to drag and drop multiple files onto TheWordBible2MySword.exe, TheWordCommentary2MySword.exe, TheWordDictionary2MySword.exe and TheWordBook2MySword.exe to do multiple conversions. If there is an error, a Press any key message will appear and pause the execution at the end.
      2. In the console, it is possible to use the wildcard * and ? but it only works on the current directory. For example, to convert all *.ont, *.nt and *.ot files in the current folder, you can do:

             TheWordBible2MySword.exe *.ont *.nt *.ot

         Similarly, you can convert all The Word dictionaries in the current directory using this:

             TheWordDictionary2MySword.exe *.dct.twm

      3. Commentary, dictionary and book migrators now included the following fields in the migration if present (displayed in MySword About Module): publisher, author, creator, source, language and editorial comments.
   3. RTF2MySwordHTML.dll RTF conversion improvements:
      1. Support for \s1 to \s6 for heading h1 to h6.
      2. Simple support for tables. The merging of rows and cells are not supported so the final output might need to be manually adjusted.
      3. Embedded font support by using span and CSS class selectors. The names of the fonts must be specified in the config's Fonts appSettings entry which must be comma separated and the corresponding CSS class names in FontClasses. TheWordDictionary2MySword.exe.config gives a commented out example which was used for AHLB module. If these config settings are present, a customCSS field is created in the Details table but the embedding of the fonts themselves must be done manually (please see AHLB.dct.mysword.zip). Note also that some old fonts cannot be rendered via @font-face so they must be converted. For example, the original font files for AHLB must be opened then regenerated using FontForge in order for them to work in HTML/CSS as well as MySword.
      4. MLSJ/AMGL conversion of special Hyperlink tw://*.*?txt=Entry or just plain double quotes delimited string will be properly converted as a Note popup similar to Translators' Notes. MySword 5.1 will properly display the Module abbreviation as title.

4. ### What's New in Version 2.3

   1. RTF2MySwordHTML.dll improvements:
      1. MLSJ conversion of special Hyperlink tw://*.*?txt=Entry will be properly converted. Currently, this is just converted to a self-link which actually means there is no entry in MLSJ as only Greek entries exist.
      2. bold/italic wrapping hyperlinks are now properly converted.
      3. initial support for RVF bullets and numbers lists including nesting. New lines (breaks entered in TheWord via Shift-ENTER) in items are also supported.
      4. support for Bible links without tags in RTF conversion if the format is BookAbbr_Chapter_Verse(range), e.g. Joh_3:16. This is because copy-paste of links in e-Sword Study Notes then migrate to TheWord commentary may not have link tags.
      5. RTF fonttbl whose list are separated by new lines are now ignored.
      6. RTF Bible links will now use the custom text link for display so that you can now see Bible links like "Ps lxx 1" as in Matthew Henry.
      7. RVF dictionary links without tid= is now supported.
      8. RTF dictionary links will now use the custom text link for display.
      9. RTF commentary self links will now work and not linked to dictionary entry.
   2. Updated TheWordBible2MySword.exe to include the publish.date if present in the TheWord file. Also made the logic that extracts information at the end of the TheWord file case-insensitive (some modules use About instead of about).
   3. Updated the dictionary migrator so that empty entries are excluded and will no longer throw an exception of Specified cast is not valid.
   4. Updated the commentary migrator so that the record that cause an error will be indicated so that it can be rectified.

5. ### What's New in Version 2.2

   1. Initial support for RTL for Commentary, Dictionary and Book migration. The r2l field is now copied to a new field in the details table righttoleft and used by MySword 2.0 RC3 for proper alignment. However, the Android device must have the patch that manufacturers normally provided for their Arabic and Hebrew customers or the Android version must be at least Honeycomb.
   2. Updated RTF2MySwordHTML.dll so that the conversion of the RFT Unicode escapes of the form \u9999 will also ignore the possibly succeeding space. This will remove the unnecessary space in between Greek letters in Strong's dictionary.
   3. RTF fonttbl (font table) support. An RTF content not using Unicode escapes (old RTF version) but with Greek, Hebrew and Baltic character sets can now be properly converted (will not use one global character set). The makers of the K&D and other modules actually stripped off the font table from the content (maybe to reduced the size) so there is now a default list of fonts:
      - font 0 (default) - Windows-1252
      - font 1 - Greek - Windows-1253
      - font 2 - Hebrew - Windows-1255
   4. Dictionary originally in RTF format converted to RVF via TheWord's "Detect all verse references..." still has data in Text data type causing no data to be migrated using the old migrator.
   5. Fixed the RTF migration of Bible reference with verse 0 and dropped the :0, e.g. Psa 92:0 will become Psa 92, as reported by Peter Pellerin.

6. ### What's New in Version 2.1

   Fixed picture extraction bug in RTF2MySwordHTML.dll for RVF content whose embedded picture contained extra RVF Strings (used for height and width). Morrish Galilee does not show as reported by a user because the image is invalid due to extra bytes at the beginning. The American Tract dictionary is also affected. Here are the entries affected:

   1. Morrish
      - Accad
      - Embalming
      - Fig, Fig-tree
      - Galilee
      - Seasons - two pictures at the end
   2. American Tract
      - Idol, Idolatry
      - Sepulchre - 3rd picture
      - Shalmaneser

7. ### What's New in Version 2.0

   1. Migration of TheWord Book, Devotional and Graphics Modules. The modules created require at least MySword version 2.0.
   2. Numerous fixes in RTF2MySwordHTML.dll's RTF and RVF conversion routines. Notably, the problem with Hebrew fonts in Barnes, some missing sections in Clarke, About/Comments in Bible having \'XX where X is a hexadecimal number are fixed. The appropriate Character Encoding to use for RTF's \'XX must be set in the *.config file of the specific migrator EXE.
   3. Graphics support for Books/Journals (to make the Maps possible), Dictionaries (Morrish now grew from being 4MB to 18MB due to pictures, and we now have American Tract Society Bible Dictionary with pictures) and Commentaries (have not yet tested though).
   4. Initial support for RTL for Bible migration. The r2l field is now copied to details table's RightToLeft and used by MySword 2.0 for proper alignment. However, the Android device must have the patch that manufacturers normally provided for their Arabic and Hebrew customers.

8. ### What's New in Version 1.2

   1. Corrected some bugs in the RTF2MySwordHTML.dll used by TheWordDictionary2MySword and TheWordCommentary2MySword.
   2. Trimmed also the Dictionary subject some words in French Strong's of Daniel Poulin have trailing space so a strong link tap from the Bible may not find an entry.
   3. Also, updated all migrators except Bible so that it will prompt the user if the module is compressed or encrypted and cannot be migrated.

9. ### What's New in Version 1.1

   TheWordDictionary2MySword is updated to support self references or links to entries in the same dictionary module using ID numbers instead of word key. Version 1.0 only supports word keys and Strong's numbers. The need arises when migrating Noah Webster's 1828 Dictionary of American English.

10. ### Installation

    This program is designed to run in Microsoft Windows with .NET Framework 3.5. It has not yet been tested in Mono Framework.

    Simply unzip the archive file to a certain location.

    For 64-bit Windows, please copy the System.Data.SQLite.DLL file located at the x64 folder into the main folder overwriting the same file there (which is actually for 32-bit Windows).

11. ### Migrating TheWord Bible Modules

    Open a Command Prompt (CMD.exe) and go to the location where you unzipped the archive file and make sure you go to the inside folder (MySwordMigrationTools).

    In the Command Prompt, if the module to be migrated is bbe.ont and is placed in the same folder where the executables are, type the following:

        TheWordBible2MySword bbe.ont

    This will create bbe.bbl.mysword. You can now just copy this file into your selected Modules Path/Folder in your Android device.

    Please note that the TheWord .ot, .nt or .ont file must be in UTF-8 format for the migration to be successful unless only 7-bit ASCII characters are in the file. To determine if the file is in UTF-8 format, you can open it in Notepad++ and go to the Encoding menu. To change the format to UTF-8 format, select Encoding->Convert to UTF-8 without BOM, then save the file. You can also opt to use UTF-8 with BOM if you want. After this, you can now use TheWordBible2MySword.

    You can also check the resulting SQLite database outside MySword by using Firefox SQLite Manager plugin/extension. Open the migrated file and check the contents by going to Bible table and selecting Browse & Search tab. Check if there are no problems with the text.

12. ### Migrating TheWord Commentary Modules

    Open a Command Prompt (CMD.exe) and go to the location where you unzipped the archive file and make sure you go to the inside folder (MySwordMigrationTools).

    In the Command Prompt, if the module to be migrated is tsk.cmt.twm and is placed in the same folder where the executables are, type the following:

        TheWordCommentary2MySword tsk.cmt.twm

    This will create tsk.cmt.mysword. You can now just copy this file into your selected Modules Path/Folder in your Android device.

    You can also use Firefox SQLite Manager plugin/extension to see the output SQLite database without using MySword. Check the Commentary table.

13. ### Migrating TheWord Dictionary Modules

    Open a Command Prompt (CMD.exe) and go to the location where you unzipped the archive file and make sure you go to the inside folder (MySwordMigrationTools).

    In the Command Prompt, if the module to be migrated is eastons.dct.twm and is placed in the same folder where the executables are, type the following:

        TheWordDictionary2MySword eastons.dct.twm

    This will create eastons.dct.mysword. You can now just copy this file into your selected Modules Path/Folder in your Android device.

    You can also use Firefox SQLite Manager plugin/extension to see the output SQLite database without using MySword. Check the Dictionary table.

14. ### Migrating TheWord Book/Devotional/Graphics Modules

    TheWord General Book (*.gbk.twm) Devotional (*.dev.twm) and Maps/Graphics (*.map.twm) all have the same format and can be migrated to MySword v.2.0 Journal or Book format (*.bok.mybible). This migration program will automatically use the extention .bok.mybible but you can always rename it to .jor.mybible. Also, the migration program will automatically set the readonly field in the details table to 1. If you intend to allow editing of the journal/book in MySword, you will have to set it to 0 manually (please see below on how you can use Firefox and SQLite manager to do that). But please note that Journal/Book editing in MySword is limited to 7 entries if the Premium Version is not activated. This can be activated for a generous donation of any amount to support MySword development, hosting and support.

    Open a Command Prompt (CMD.exe) and go to the location where you unzipped the archive file and make sure you go to the inside folder (MySwordMigrationTools).

    In the Command Prompt, if the module to be migrated is eastons.dct.twm and is placed in the same folder where the executables are, type the following:

        TheWordBook2MySword wight-manners.gbk.twm

    This will create eastons.dct.mysword. You can now just copy this file into your selected Modules Path/Folder in your Android device.

    You can also use Firefox SQLite Manager plugin/extension to see the output SQLite database without using MySword. Check the Dictionary table.

15. ### Migration Errors

    Note that Bible References that do not have correct book number and chapter number will not be properly migrated (so there will be no link) but the migration process will still continue. The errors are reported to the console though.

16. ### Known Issues

    The migration program will crash in case of IO Exceptions like the one triggered when the .mysword file is opened in Firefox SQLite Manager or you are running the wrong version of System.Data.SQLite.DLL (32-bit in a 64-bit Windows for example).