2019-01-25T10:29:22Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*'''1st step: Fix orientation'''. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.

<gallery>
File:Rotate.png
</gallery>

Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*'''2nd step: Split pages'''. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.

<gallery>
File:SplitPages.png
</gallery>

[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*'''3rd step: Deskew'''. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal.
<gallery>
File:Deskew main tab.jpeg
</gallery>

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*'''4th step: Select content'''. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.

<gallery>
File:Content main tab.jpeg
</gallery>

[Select content manual -> https://github.com/scantailor/scantailor/wiki/Select-Content]

*'''5th step: Margins'''. Check out all margins so they are set to zero. Then, place the content in a manner that will help it being read "centralized".

<gallery>
File:Margins to zero.png
</gallery>

<gallery>
file:Center.png
</gallery>

[Margins management manual -> https://github.com/scantailor/scantailor/wiki/Page-Layout]

*'''6th step: Output'''. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider. Check every tab on the column that emerged on the right:
* Picture Zones: use the tool to select areas with photographs, illustrations or special icons.
* Dewarping: manage the grid to stregthen your page's content.
* Desplekling: remove dust and dots from the page.

[Output manual -> https://github.com/scantailor/scantailor/wiki/Output]

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-25T09:59:38Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*'''1st step: Fix orientation'''. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.

<gallery>
File:Rotate.png
</gallery>

Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*'''2nd step: Split pages'''. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.

<gallery>
File:SplitPages.png
</gallery>

[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*'''3rd step: Deskew'''. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal.
<gallery>
File:Deskew main tab.jpeg
</gallery>

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*'''4th step: Select content'''. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.

<gallery>
File:Content main tab.jpeg
</gallery>

[Select content manual -> https://github.com/scantailor/scantailor/wiki/Select-Content]

*'''5th step: Margins'''. Check out all margins so they are set to zero. Then, place the content in a manner that will help it being read "centralized".

<gallery>
File:Margins to zero.png
</gallery>

<gallery>
file:Center.png
</gallery>

[Margins management manual -> https://github.com/scantailor/scantailor/wiki/Page-Layout]

*'''6th step: Output'''. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-25T09:56:05Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*'''1st step: Fix orientation'''. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.

<gallery>
File:Rotate.png
</gallery>

Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*'''2nd step: Split pages'''. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.

<gallery>
File:SplitPages.png
</gallery>

[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*'''3rd step: Deskew'''. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal.
<gallery>
File:Deskew main tab.jpeg
</gallery>

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*'''4th step: Select content'''. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.

<gallery>
File:Content main tab.jpeg
</gallery>

[Select content manual -> https://github.com/scantailor/scantailor/wiki/Select-Content]

*'''5th step: Margins'''. Check out all margins so they are set to zero. Then, place the content in a manner that will help it being read "centralized".

<gallery>
File:Margins to zero.png
</gallery>

<gallery>
file:Center.png
</gallery>

*'''6th step: Output'''. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

File:Center.png

2019-01-25T09:55:03Z

Jxxx:

Bookscanning

2019-01-25T09:54:26Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*'''1st step: Fix orientation'''. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.

<gallery>
File:Rotate.png
</gallery>

Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*'''2nd step: Split pages'''. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.

<gallery>
File:SplitPages.png
</gallery>

[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*'''3rd step: Deskew'''. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal.
<gallery>
File:Deskew main tab.jpeg
</gallery>

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*'''4th step: Select content'''. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.

<gallery>
File:Content main tab.jpeg
</gallery>

[Select content manual -> https://github.com/scantailor/scantailor/wiki/Select-Content]

*'''5th step: Margins'''. Check out all margins so they are set to zero. Then, place the content in a manner that will help it being read "centralized".

<gallery>
File:Margins to zero.png
</gallery>

<gallery>

</gallery>

*'''6th step: Output'''. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-25T09:53:00Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*'''1st step: Fix orientation'''. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.

<gallery>
File:Rotate.png
</gallery>

Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*'''2nd step: Split pages'''. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.

<gallery>
File:SplitPages.png
</gallery>

[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*'''3rd step: Deskew'''. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal.
<gallery>
File:Deskew main tab.jpeg
</gallery>

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*'''4th step: Select content'''. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.

<gallery>
File:Content main tab.jpeg
</gallery>

[Select content manual -> https://github.com/scantailor/scantailor/wiki/Select-Content]

*'''5th step: Margins'''. Check out all margins so they are set to zero. Then, place the content in a manner that will help it being read "centralized".

<gallery>
File:Margins to zero.png
</gallery>

<gallery>
File:Center.png
</gallery>

*'''6th step: Output'''. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-25T09:51:50Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*'''1st step: Fix orientation'''. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.

<gallery>
File:Rotate.png
</gallery>

Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*'''2nd step: Split pages'''. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.

<gallery>
File:SplitPages.png
</gallery>

[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*'''3rd step: Deskew'''. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal.
<gallery>
File:Deskew main tab.jpeg
</gallery>

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*'''4th step: Select content'''. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.

<gallery>
File:Content main tab.jpeg
</gallery>

[Select content manual -> https://github.com/scantailor/scantailor/wiki/Select-Content]

*'''5th step: Margins'''. Check out all margins so they are set to zero. Then, place the content in a manner that will help it being read "centralized".

<gallery>
File:Margins to zero.png
</gallery>

*'''6th step: Output'''. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

2019-01-24T19:26:04Z

Jxxx:

Bookscanning

2019-01-24T19:24:50Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*1st step: Fix orientation. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.
<gallery>
File:Rotate.png
</gallery>
Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*2nd step: Split pages. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.
<gallery>
File:SplitPages.png
</gallery>
[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*3rd step: Deskew. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal
[[File:Deskew main tab.jpeg|thumb]]

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*4th step: Select content. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.

[Select content manual -> https://github.com/scantailor/scantailor/wiki/Select-Content]

*5th step: Margins. Check out all margins place the content in a manner that will help it being read "centralized".
*6th step: Output. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-24T19:22:30Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*1st step: Fix orientation. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.
<gallery>
File:Rotate.png
</gallery>
Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*2nd step: Split pages. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.
<gallery>
File:SplitPages.png
</gallery>
[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*3rd step: Deskew. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal
[[File:Deskew main tab.jpeg|thumb]]

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*4th step: Select content. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.
*5th step: Margins. Check out all margins place the content in a manner that will help it being read "centralized".
*6th step: Output. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

File:Deskew main tab.jpeg

2019-01-24T19:21:14Z

Jxxx:

Bookscanning

2019-01-24T19:20:40Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*1st step: Fix orientation. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.
<gallery>
File:Rotate.png
</gallery>
Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*2nd step: Split pages. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.
<gallery>
File:SplitPages.png
</gallery>
[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]

*3rd step: Deskew. Drag and determine the angle which the page needs to be turned for the text and images to be properly horizontal

[Deskew manual -> https://github.com/scantailor/scantailor/wiki/Deskew]

*4th step: Select content. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.
*5th step: Margins. Check out all margins place the content in a manner that will help it being read "centralized".
*6th step: Output. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-24T19:17:05Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*1st step: Fix orientation. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.
<gallery>
File:Rotate.png
</gallery>
Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]

*2nd step: Split pages. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.
<gallery>
File:SplitPages.png
</gallery>
[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]
*3rd step: Deskew. Determine the angle which the page needs to be turned for the text and images to be properly horizontal
*4th step: Select content. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.
*5th step: Margins. Check out all margins place the content in a manner that will help it being read "centralized".
*6th step: Output. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-24T19:15:37Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*1st step: Fix orientation. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.
<gallery>
File:Rotate.png
</gallery>
Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".

[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]
*2nd step: Split pages. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.
<gallery>
File:SplitPages.png
</gallery>
[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]
*3rd step: Deskew. Determine the angle which the page needs to be turned for the text and images to be properly horizontal
*4th step: Select content. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.
*5th step: Margins. Check out all margins place the content in a manner that will help it being read "centralized".
*6th step: Output. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

Bookscanning

2019-01-24T19:14:41Z

Jxxx: /* Postproduction */

There are many ways to scan, this is the current state of the art in Calafou. We use only free software and the documentation is for Debian GNU/Linux, but it should work with some small modifications on any UNIX based system running the bash shell. There are some parts where proprietary software such as ABBYY FineReader can be more effective. However, this workflow produces near perfect books in PDF format that we are very happy with. One thing we could definitely improve is the size of the final PDF file, which is quite big (can be more than 100 megabytes).

= Scanning =

'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''

# Setting up the cameras (calibration): the most important part.

* open the book in the middle (at a central page with normal text on both sides)
* '''camera should look directly on the middle of the page, parallel to the cradle, at 45 degrees compared to horizontal'''
* all the page should be in the image, but it is not a problem if more things outside of the book are visible
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book…)
* camera settings: fully automatic, perhaps with manual focus
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera (this can be a difference in the zoom level or the distance between camera and page)
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning

<ol start="2" style="list-style-type: decimal;">
<li>Push the big button on the scanner to scan.</li></ol>

* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages

<ol start="3" style="list-style-type: decimal;">
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>

* from the camera on the left, copy the images to a folder called “odd”
* from the camera on the right, copy the images to a folder called “even”
* upload the two folders now to to <code>ftp://omnius.calafou/HackTheBiblio/scanning/$bookname--$yourname/</code> folder
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge

'''Additional information using the Marron scanner'''

* Check before starting that the SD card are locked: the external trigger that controls the cameras requires the SD cards to be locked. If they are not locked, the pictures are not saved when using the external trigger.
* Camera settings: we use two IXUS 175 set to automatic with menu/lamp setting set to "off" to avoid the use of the red light
* While taking pictures, if you need to check the last picture taken: long press the green play button to enter slideshow mode, long press the green play button to go back to picture mode (half pressure on the camera trigger also works)
* If you decide to use the zoom of the camera (not the digital zoom), be careful not to turn off the camera or you will loose your zoom setting

= Dependencies =

Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:

* scantailor
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre

You can install all these programs with the following invocation from the command line (also called the terminal):

<pre>sudo apt install scantailor gprename pdftk tesseract-ocr tesseract-ocr-eng tesseract-ocr-spa calibre</pre>

= Postproduction =

You start with two folders such as <code>odd</code> and <code>even</code> with files like IMG_1234.JPG. It is not good to talk about <code>right</code> and <code>left</code> because it can be very confusing: are you talking about the image from the right camera that takes pictures of the left page of the book, or the image of the left page of the book that is from the right camera? On the other hand, <code>odd</code> (1, 3, 5, …) and <code>even</code> (2, 4, 6, …) are good words for describing what is on the image without ambiguity!

The basic workflow is like this:

<ol start="0" style="list-style-type: decimal;">
<li> [process] → [program] → [output]</li>
<li> Merge pictures from the two cameras → gprename → 1.jpg, 2.jpg, …</li>
<li> Edit the pictures to adjust contents → scantailor → 1.tif, 2.tif, …</li>
<li> Character recognition → tesseract → 1.pdf, 2.pdf, …</li>
<li> Create the pdf file → pdftk → book.pdf</li>
<li> Create the ebook → calibre → book.epub</li>
<li> Disseminate → libgen.org → http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>

There are some bash oneliners which can be useful (on Debian based systems):

<ol style="list-style-type: decimal;">
<li> <code>gprename</code>
* enter <code>gprename</code> using a Terminal
* go to the Directory with the odd files
* select all files
* go to the numerical tab
* set starting number to 1 and increment by 2
* set the naming pattern

[[File:Gprename.png|gprename window for renaming files]]

* repeat the operation for even files
* merge the two folders
* <code>FIXME</code> we can probably write a script to rename the files properly…</li>
<li> <code> scantailor </code>

You can edit the captures appropriately with [http://scantailor.org/ scantailor]. It invites you to follow these steps:

*1st step: Fix orientation. All odd pages need to be turned in one direction, while even pages need to be turned in the other direction.
<gallery>
File:Rotate.png
</gallery>
Rotate image nr 1 and click on "apply to every other page". Then select image nr 2, rotate in the opposite direction so it stays still, and also click on "apply to every other page".
[[Fix Orientation Manual -> https://github.com/scantailor/scantailor/wiki/Fix-Orientation]]
*2nd step: Split pages. If you import all files renamed, odd and even pages will be recognized as single pages, so this step is just to confirm that the edges of the pages are set properly; drag the rectangles to fit in the page's area.
<gallery>
File:SplitPages.png
</gallery>
[[Split pages manual -> https://github.com/scantailor/scantailor/wiki/Split-Pages]]
*3rd step: Deskew. Determine the angle which the page needs to be turned for the text and images to be properly horizontal
*4th step: Select content. Frame all elements to be shown as content, within one single area (beware of including for example page numbers). The outer limit of these margins affects the size of the output file.
*5th step: Margins. Check out all margins place the content in a manner that will help it being read "centralized".
*6th step: Output. Consider the visibility/readability of pages with images and/or mixed img-txt, managing the thickness slider.

<li>Does Optical Character Recognition (OCR) on all images in folder:
<pre> time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li>Merges all the pdf files in folder into one single file:
<pre> pdftk *pdf cat output book.pdf</pre></li>
<li>Exports the pdf metadata to a text file, to edit:
<pre> pdftk book.pdf dump_data output report.txt</pre></li>
<li>Imports the metadata of report.txt back to the PDF:
<pre> pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>

= Distribution =

Think about how people who would be interested in this book could know about it!

Repositories:

* General “educational materials”: [https://libgen.io/ Library Genesis]
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
* Artist radical: [https://monoskop.org/ Monoskop]
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…

You may consider spreading the word on relevant mailing lists, social media, etc.

= Biblio-graphy =

* [https://www.memoryoftheworld.org/wp-content/uploads/2014/12/scanning_manual_v1.2.pdf Scanning Manual from Memory of the World]: a quite long document in PDF
* [https://www.memoryoftheworld.org/ Memory of the World]: Digital Public Libraries
* [https://www.memoryoftheworld.org/es/ Spanish pages on Memory of the World]: Digital Public Libraries in Spanish
* [http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]: Book digitalisation manual

File:SplitPages.png

2019-01-24T19:11:47Z

Jxxx: