Bookscanner: Difference between revisions

From Wiki-Fou
(Add section on budget)
(Add note about "document cameras")
(41 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Kunlabora bookscanner building in progress right now! :) We are building this: https://forum.diybookscanner.org/viewtopic.php?f=1&t=333
Bookscanning in Calafou:


= Building a scanner =
* We made an event called Hack the Biblio: Let's make Digital Public Libraries in 2014 as a collaboration with memoryoftheworld, when Voja Antonic donated a bookscanner to Calafou: https://calafou.org/en/content/hack-biblio-construir-bibliotecas-publicas https://calafou.org/en/content/voja-antonic-calafou
* We scan out of print books published by Virus: http://www.viruseditorial.net/
* <span style='width:32em;font-size:18px;font-weight:light;font-family:Courier,Sans;display:block;padding:4px;background:#DAA520;color:black;'>marron_scanner 😎 </span> was built in 2018 during the Kunlabora event for supporting other collectives that want to scan: https://calafou.org/en/content/kunlabora-ephimeral-projects-kooperative


Here are two links to the public documentation of our scanner, built by Voja Antonic:
This page is documenting the design and construction of bookscanners.  For the tutorial that explains "How to use the bookscanner", go to http://wiki.calafou.org/index.php/Bookscanning
 
= Building a "New Standard Scanner" =
 
We built this: https://forum.diybookscanner.org/viewtopic.php?f=1&t=333 during the Kunlabora event in Calafou: https://calafou.org/en/content/kunlabora-ephimeral-projects-kooperative A more reproducible and portable and simple scanner.
 
In theory for the simple scanner it should be possible to get all the ingredients from a hardware shop ("ferreteria").  We got most of the materials from Bauhaus in Barcelona (Zona Franca).  Our experience was that most of the screws, nuts and bolts are easy to substitute with others that have more or less the same dimensions.
 
[[File:bookscanner_pseudoready2.jpg|600px]]
 
== Budget ==
 
Kunlabora event was four days of collaborative construction.  More or less six persons worked on building the scanner.  This is the receipts that we spent, because we had 500 EUR budget for the raw materials from the participation fee of the event.


https://www.memoryoftheworld.org/blog/2012/10/28/our-beloved-bookscanner-2/
Receipts:


https://hackaday.io/project/5604-diy-book-scanner
    25    2x plexiglass [J] @ Expocryl, Igualada
    18    2x SD cards @ Life Informatica, Barcelona
    50    2x magic want camera stand @ FotoK, Barcelona
    42.2  Nuts/bolts/etc. @ Bauhaus, Zona Franca, Barcelona
    108.8 Nuts/bolts/etc. @ Bauhaus, Zona Franca, Barcelona
    10    2x USB cable male/female @ Worten San Antoni, Barcelona
    5    2x mini USB (USB-B) cable @ Tienda Cables, Barcelona
    178  2x Canon IXUS 175 compact digital camera @ Amazon.es
    60    4x Wooden beams (listones) @ Fustes Fargas, Barcelona
    ============================================
    497


We are trying to build something similar during the Kunlabora event in Calafou: https://calafou.org/en/content/kunlabora-ephimeral-projects-kooperative
Donations:


We decided to build a more reproducible and portable and simple scanner: https://forum.diybookscanner.org/viewtopic.php?f=1&t=333
    70    Raspberry Pi 3 (Model B), 2x SD cards, 5V charger, rulers [M] @ Barcelona


In theory for the simple scanner it should be possible to get all the ingredients from a hardware shop ("ferreteria").  We got most of the materials from Bauhaus in Barcelona (Zona Franca)TODO: list of materials, sources and prices.
We also used nuts and bolts, etc. found in the workshop of Calafou (in Catalunya, near Barcelona).  The conclusion is that it should be possible to build this scanner from about 5-600 EUR anywhere in (Western) Europe.  We did not use all the materials we bought and if we would do it again then we tried to use real wood instead of FDM, because the latter proved very fragileHowever, this could make the platen more heavy, and it is already quite heavy.


== Ideas for the building Voja's scanner ==
== Parts / componentes ==


The electronics is not really documented (which means that it is hard to reproduce) and it is built from basic parts (which means that it takes a lot of time to put it together.  So we try to use an Arduino-based solution instead.  Arduino is a general-purpose programmable microcontroller that has already built-in many of the functions/parts we need.  The idea is that this makes it easier for us to build the scanner and for others to reproduce it.  We also have more experience working with Arduino than with only basic electronic components.


We will also try to use cheaper cameras in order to bring down the budget.
[[File:01 SHOPPING 001.JPG|500px|none|left]]
<i>Going for shopping</i>
[[File:01 SHOPPING 003.JPG|500px|none|left]]


== List of parts ==


From Bauhaus, Barcelona (Zona Franca):
<b>From Bauhaus, Barcelona (Zona Franca):</b>


  - 2x Tablero MDF 600x400x10cm
[[File:01 SHOPPING 027.JPG|500px|none|left]]
  - 2x Tablero MDF 600x400x16cm
-1x Varilla roscada
- Varilla roscada
-4x Listones de madera  ~800x40x90mm
  - 2x Tira LED 1 nice
  -2x (Tablero) MD sheet 600x400x10mm
- Multienchufa 5 tomas
  -2x (Tablero) MDF sheet 600x400x16mm
[[File:01 SHOPPING 023.JPG|500px|none|left]]
  - Cola blanca rápida para madera
[[File:01 SHOPPING 024.JPG|500px|none|left]]
  - Base 4 tomas 1.5 mts
  - Base 4 tomas 1.5 mts
[[File:01 SHOPPING 007.JPG|500px|none|left]]
  - Tuerca hexagonal
  - Tuerca hexagonal
[[File:01 SHOPPING 020.JPG|500px|none|left]]
  - Guia corredera
  - Guia corredera
  - 2x Arandela ancha
[[File:01 SHOPPING 004.JPG|500px|none|left]]
  - Arandelas ancha
[[File:01 SHOPPING 005.JPG|500px|none|left]]
  - Aldabon BPF 185 M
  - Aldabon BPF 185 M
[[File:01 SHOPPING 006.JPG|500px|none|left]]
  - Tornillo mad.cab.pl.
  - Tornillo mad.cab.pl.
[[File:01 SHOPPING 010.JPG|500px|none|left]]
  - Tornillo cab. avell.
  - Tornillo cab. avell.
[[File:01 SHOPPING 011.JPG|500px|none|left]]
  - Tornillo p/metal
  - Tornillo p/metal
[[File:01 SHOPPING 008.JPG|500px|none|left]]
  - Tuerca palomilla
  - Tuerca palomilla
[[File:01 SHOPPING 021.JPG|500px|none|left]]
  - Guia cajones 10kg
  - Guia cajones 10kg
[[File:01 SHOPPING 022.JPG|500px|none|left]]
  - Spax cab. red zinc
  - Spax cab. red zinc
[[File:01 SHOPPING 012.JPG|500px|none|left]]
  - Tirador dorado
  - Tirador dorado
 
[[File:01 SHOPPING 014.JPG|500px|none|left]]
  - 4x Abrazaderas
  - 4x Abrazaderas
- Colgador acero negro
[[File:01 SHOPPING 025.JPG|500px|none|left]]
  - Mosqueton-fw
  - Mosqueton-fw


Other things from other shops:
- 2 x plexi glass 380x280x3mm from a company in Igualada that J. found.


  - 3x Cable USB 2.0 male-female @ Worten Sant Antoni
<b>Other things from other shops:</b>
 
[[File:01 SHOPPING 013.JPG|500px|none|left]]
  - 4x Cable USB 2.0 male-female @ Worten Sant Antoni
  - 2x USB-A -> USB-B (mini) cables @ Tienda de Cables
  - 2x USB-A -> USB-B (mini) cables @ Tienda de Cables
- 2x SD Cards 16 Gg
[[File:01 SHOPPING 017.JPG|500px|none|left]]
  - 2x Manfrotto brazo flexible MF237 (Nr. referencia AFP018036) @ FotoK, Ronda Universitat
  - 2x Manfrotto brazo flexible MF237 (Nr. referencia AFP018036) @ FotoK, Ronda Universitat
[[File:01 SHOPPING 015.JPG|500px|none|left]]
- 2x Tira LED 1 nice
  - 2 x digital cameras (see below for background information): Canon IXUS 175 (Powershot / ELPH 180)
  - 2 x digital cameras (see below for background information): Canon IXUS 175 (Powershot / ELPH 180)
   - https://www.amazon.es/Canon-IXUS-175-compacta-estabilizador/dp/B01A8QU70I/
   - https://www.amazon.es/Canon-IXUS-175-compacta-estabilizador/dp/B01A8QU70I/
   - http://chdk.wikia.com/wiki/ELPH180
   - http://chdk.wikia.com/wiki/ELPH180
   - https://www.canon-europe.com/for_home/product_finder/cameras/digital_camera/ixus/ixus_175/specification.aspx
   - https://www.canon-europe.com/for_home/product_finder/cameras/digital_camera/ixus/ixus_175/specification.aspx
- 2 x gooseneck camera stand ("magic wand")
- 2 x plexi glass from a company in Igualada that J. found.


For electronic parts a good shop in Barcelona is Diotronic: https://diotronic.com/
For electronic parts a good shop in Barcelona is Diotronic: https://diotronic.com/


== Budget ==
  - usb charger to give power to the external trigger trough USB cable
 
Kunlabora event was four days of collaborative construction. More or less six persons worked on building the scanner.  This is the receipts that we spent, because we had 500 EUR budget for the raw materials from the participation fee of the event.


Receipts:
<b>Things to take in consideration from the building experience</b>


    25    2x plexiglass [J]
* the MDF pieces are fragile and gave us problems when introducing screws
    18    2x SD cards @ Life Informatica
* we considered varnishing the wood pieces to limit humidity impact
    50    2x magic want camera stand @ FotoK
* the plexiglas we bought generates many reflections
    42.2  Nuts/bolts/etc. @ BAUHAUS
* passing from inches to cm was uncomfortable
    108.8 Nuts/bolts/etc. @ BAUHAUS
* the position and height of the light should be tested before being fixed
    10    2x USB cable male/female @ Worten
* the upper part of the light stand should be easier to mount and unmount for transportation
    5    2x mini USB (USB-B) cable @ Tienda Cables
* the wood used for the platen should be without texture or painted with mate dark paint
    178  2x Canon IXUS 175 compact digital camera
* the gooseneck (Manfrotto) option for the camera stand is not a good option: it is very easy to put the camera in the way of the cradle; the calibration process is difficult because the cameras can move in the three axis; they are springy and difficult to set in an exact position
    60    4x Listones @ Fustes Fargas
    ============================================
    497
 
Donations:
 
    70    Raspberry Pi 3 (Model B), 2x SD cards, 5V charger, rulers [M]
 
We also used nuts and bolts, etc. found in the workshop of Calafou (in Catalunya, near Barcelona).  The conclusion is that it should be possible to build this scanner from about 600 EUR in (Western) Europe.  We did not use all the materials we bought and if we would do it again then we tried to use real wood instead of FDM, because the latter proved very fragile.


== Cameras ==
== Cameras ==


Summary of research about cameras for book scanners:
The bookscanner is basically a glorified camera stand that allows us to take good pictures of book pages.  So the cameras are an important part of the bookscanner.  Summary of research about cameras for book scanners:


Basically there are three categories of cameras that can be used for book scanners (from cheapest to most expensive).
Basically there are three categories of cameras that can be used for book scanners (from cheapest to most expensive).
Line 97: Line 130:
2. CHDK firmware
2. CHDK firmware


Middle category is CHDK firmware compatibles.  CHDK is a third party open source firmware that allows the customisation of cameras.  CHDK firmware is for Canon Powershot cameras, which are the cheaper compact digital camera product line.  We have 200 euros in the budget for cameras, so we went with this option.
Middle category is CHDK firmware compatibles.  CHDK is a third party open source firmware that allows the customisation of cameras.  CHDK firmware is for Canon Powershot cameras, which are the cheaper compact digital camera product line.  We had 200 euros in the budget for cameras, so we went with this option.


3. Magic Lantern support
3. Magic Lantern support
Line 103: Line 136:
Magic Lantern is a third party open source firmware that is more advanced.  However, it only works with Canon DLSR cameras (these are the cameras that have a reflex mirror to look at the shot through a small hole before you take the picture, and they usually have big lenses). The scanner we have now uses Canon 1100D, which are the cheapest type suported by Magic Lantern, but they still cost a few hundred euros.
Magic Lantern is a third party open source firmware that is more advanced.  However, it only works with Canon DLSR cameras (these are the cameras that have a reflex mirror to look at the shot through a small hole before you take the picture, and they usually have big lenses). The scanner we have now uses Canon 1100D, which are the cheapest type suported by Magic Lantern, but they still cost a few hundred euros.


4. Connecting to a monitor
It is very useful to be able to connect the cameras to a monitor during the calibration process. The cameras we bought use the same output for connecting to a remote control and to a monitor making it not practical at all using a monitor. Furthermore, connection with usb to a computer monitor was not an option, we had to look for RC connectors or adapters.
5. A note about webcams:
An option might seem to use two high resolution webcams directly connected to a laptop to trigger taking pictures and monitor their quality, and there are full HD webcams (~2 Megapixels). The problem with all webcams, however, is that they have a fixed optical zoom, at quite a wide angle. To get those pixels covering the book page, you need to be very close to the book (~25cm). This forces you to use a design similar to what Reece Arnott uses, or the Ion Audio Booksaver. These designs make it impossible to avoid glare from light sources, no matter where the lights are placed. To avoid glare, you need a narrow zoom and a good distance (~90 cm) between book and camera.
6. Follow up: document cameras
The digital cameras mentioned above proved unreliable on the long run, because it was difficult/impossible to make them focus on the page.  It seems this is a problem with this particular model, reported by many users on forums.  The solution was to invest more money and buy specialised "document cameras", such as this one: <https://www.ipevo.com/product/v4k/gallery>
== Building process ==
=== Base / Base ===
[[File:02 BASE 035.JPG|500px|none|left]]
[[File:02 BASE 043.JPG|500px|none|left]]
[[File:02 BASE 040.JPG|500px|none|left]]
We cut the wood beams to have 2 pieces of 56x4cm and 2 of 31.5x4cm
[[File:02 BASE 044.JPG|500px|none|left]]
[[File:02 BASE 041.JPG|500px|none|left]]
we measured with precision tools and marked the position of the drawer sliders on the 54x4cm wood beams
[[File:02 BASE 045.JPG|500px|none|left]]
we drilled holes on the marked position
[[File:02 BASE 049.JPG|500px|none|left]]
[[File:02 BASE 050.JPG|500px|none|left]]
[[File:02 BASE 051.JPG|500px|none|left]]
and then we fixed the sliders
[[File:02 BASE 033.JPG|500px|none|left]]
We used the thicker MDF material to cut a 37x29cm sheet
[[File:02 BASE 056.JPG|500px|none|left]]
The last parts were fixing the other part of the sliders to the MDF sheet and attaching the different part of the base with screws


== Pictures Day 1 (temporary) ==
<gallery>
File:IMG_20181012_161540990.jpg
File:IMG_20181012_161552147.jpg
File:IMG_20181012_161615376.jpg
File:IMG_20181012_161626209.jpg
File:IMG_20181012_161639871.jpg
File:IMG_20181012_162253725.jpg
File:Day1_3.jpg
</gallery>


=== Cradle / Cuna ===
The cradle is a V-shaped book holder partly fixed to the base
[[File:07 GEN 036.JPG|500px|none|left]]
We cut two MDF (thin) sheets for the book holder


[[File:03 CRADDLE 037.JPG|500px|none|left]]
[[File:03 CRADDLE 046.JPG|500px|none|left]]
And 4 small triangles to maintain the book holder on the base with a 90 degree angle between both sheets
[[File:03 CRADDLE 057.JPG|500px|none|left]]
We measured were to position the triangles
[[File:03 CRADDLE 071.JPG|500px|none|left]]
Drilled the MDF sheet on the base
[[File:03 CRADDLE 058.JPG|500px|none|left]]
And fixed the MDF small triangles


= Scanning =
[[File:03 CRADDLE 059.JPG|500px|none|left]]
We drilled the MDF sheet of the Craddle
[[File:03 CRADDLE 061.JPG|500px|none|left]]
And fixed it to the triangles


'''The amount of work in the postproduction phase depends on how good quality images you can make in the scanning phase!'''
NOT DOCUMENTED WITH PICTURES
[[File:05 COLUMN 066.JPG|500px|none|left]]
The 2 other triangles were fixed to the other MDF sheet without being fixed to the base.
The distance between the triangles is of 19.5cm.
We cut a 19.5x19.5cm MDF square that we attached to the other side of the base.
The drilled the 2 triangles in order to introduce a threaded rod.
The threaded rod is used to tighten the triangles against the square fixed on the base.


# Setting up the cameras (calibration): the most important part.


Caveats:
=== Platen / Platina ===


* open the book right in the middle (at the central page) to calibrate the cameras.
[[File:04 PLATEN 039.JPG|500px|none|left]]
* camera should look at right angle on the page. Make sure the cameras are parallel to the angles of the cradle.
[[File:04 PLATEN 047.JPG|500px|none|left]]
* all the page should be in the image
* check if the pages fold/curve; if so, place something underneath to straighten it (like a sponge, or another book...)
* camera settings: fully automatic, perhaps with manual focus.  
* back up and empty the SD cards in the cameras
* most subtle mistake: one camera sees letters bigger than the other camera
* use a post-it or similar to mark the exact position of the book in relation to the lower edge of the cradle, to ensure it remains in the same position throughout the scanning.
*


[[File:04 PLATEN 053.JPG|500px|none|left]]
[[File:04 PLATEN 054.JPG|500px|none|left]]
[[File:04 PLATEN 055.JPG|500px|none|left]]


<ol start="2" style="list-style-type: decimal;">
[[File:04 PLATEN 063.JPG|500px|none|left]]
<li>Push the big button on the scanner to scan.</li></ol>


* maybe you have to put your finger to the side of the plexiglass which is closer to you when it is “down”, because the plexiglass is not always exactly the same angle as the book pages


<ol start="3" style="list-style-type: decimal;">
=== Column / Columna ===
<li>Download the images from the SD cards and put the scanner to sleep.</li></ol>


* from the camera on the left, copy the images to a folder called “odd”
[[File:05 COLUMN 065.JPG|500px|none|left]]
* from the camera on the right, copy the images to a folder called “even”
[[File:05 COLUMN 066.JPG|500px|none|left]]
* upload the two folders now to to ftp://seldon.calafou/HackTheBiblio/scanning/<math>bookname--</math>yourname/ folder
[[File:05 COLUMN 067.JPG|500px|none|left]]
* remember to delete the pictures from the SD cards and put them back to the cameras, and maybe put the camera batteries to charge
[[File:05 COLUMN 068.JPG|500px|none|left]]


= Dependencies =
[[File:05 COLUMN 070.JPG|500px|none|left]]
[[File:05 COLUMN 072.JPG|500px|none|left]]
[[File:05 COLUMN 073.JPG|500px|none|left]]
[[File:05 COLUMN 074.JPG|500px|none|left]]
[[File:05 COLUMN 075.JPG|500px|none|left]]
[[File:05 COLUMN 076.JPG|500px|none|left]]
[[File:05 COLUMN 077.JPG|500px|none|left]]
[[File:05 COLUMN 078.JPG|500px|none|left]]
[[File:05 COLUMN 079.JPG|500px|none|left]]
[[File:05 COLUMN 080.JPG|500px|none|left]]
[[File:05 COLUMN 083.JPG|500px|none|left]]
[[File:05 COLUMN 084.JPG|500px|none|left]]


There are many ways to scan, this is the current state of the art in Calafou.
[[File:06 MOUNTED 086.JPG|500px|none|left]]


Using an up-to-date Debian operating system, you can install the following programs for the postproduction steps:
=== CHDK ===


* scantailor
=== External trigger ===
* gprename
* pdftk
* tesseract-ocr
* tesseract-ocr-eng
* tesseract-ocr-spa
* calibre


You can install all these programs with the following invocation:
[[File:08 TRIGERING 081.JPG|500px|none|left]]
[[File:08 TRIGERING 082.JPG|500px|none|left]]


<pre>sudo apt install scantailor gprename pdftk tesseract-ocr /
= Photos =  
        tesseract-ocr-eng tesseract-ocr-spa calibre</pre>
= Postproduction =


You start with two folders with files like IMG_1234.JPG


The basic workflow is like this:
== Mounting ==


<ol start="0" style="list-style-type: decimal;">
<gallery>
<li>[program] ➔ [output]</li>
File:06_MOUNTED_064.JPG
<li>gprename ➔ 1.jpg, 2.jpg, …</li>
File:06_MOUNTED_086.JPG
<li>scantailor ➔ 1.tif, 2.tif, …</li>
File:06_MOUNTED_088.JPG
<li>tesseract ➔ 1.pdf, 2.pdf, …</li>
File:06_MOUNTED_089.JPG
<li>pdftk ➔ book.pdf</li>
</gallery>
<li>calibre ➔ book.epub</li>
<li>libgen.org ➔ http://libgen.org/book/index.php?md5=B6916395FDE00D91DB4F52DCB8F069BF</li>
<li>etc.</li></ol>


There are some bash oneliners which can be useful (on Debian based systems):
== Miscellaneous ==


<ol style="list-style-type: decimal;">
<gallery>
<li><p><code>FIXME</code> we can probably write a script to rename the files properly… but for now, in gprename select the “numberical” tab, start = 1 for right-pages and 2 for left-pages, always step = 2.</p></li>
File:07_GEN_031.JPG
<li><p>You can rotate the images appropriately (which is called “fix orientation” in scantailor) in the left/right folders before you import them. This is faster than in scantailor I think. However, you can also make the same operation in scantailor in a more user friendly way.</p>
File:07_GEN_034.JPG
<pre>sudo apt-get install imagemagick
File:07_GEN_036.JPG
cd left
File:07_GEN_052.JPG
mogrify -verbose -rotate 270 *
File:07_GEN_085.JPG
cd ../right
</gallery>
mogrify -verbose -rotate 90 *</pre></li>
<li><p>Does Optical Character Recognition (OCR) on all images in folder:</p>
<pre>time for i in *tif; do b=$(basename $i .tif); tesseract -l spa "$i" "$b" pdf; done</pre></li>
<li><p>Merges all the pdf files in folder into one single file:</p>
<pre>pdftk *pdf cat output book.pdf</pre></li>
<li><p>Exports the pdf metadata to a text file, to edit:</p>
<pre>pdftk book.pdf  dump_data output report.txt</pre></li>
<li><p>Imports the metadata of report.txt back on the pdf:</p>
<pre>pdftk book.pdf update_info report.txt output bookcopy.pdf</pre></li></ol>


= Distribution =
== Triggering mechanism ==


Think about how people who would be interested in this book could know about it!
<gallery>
File:08_TRIGERING_069.JPG
File:08_TRIGERING_081.JPG
File:08_TRIGERING_082.JPG
</gallery>


Repositories:
== Camera holder ==


* General “educational materials”: [https://libgen.io/ Library Genesis]
<gallery>
* Public library, radical books: [https://library.memoryoftheworld.org/ Memory of the World]
File:09_CAM-HOLDERS_087.JPG
* Academic radical: [https://aaaaarg.org/ Aaaaarg]
File:09_CAM-HOLDERS_090.JPG
* Artist radical: [https://monoskop.org/ Monoskop]
</gallery>
* Anarchist (including fanzines): [https://theanarchistlibrary.org/special/index Anarchist Library]
* There are many Zine Libraries you can find on the Internet…


You may consider spreading the word on relevant mailing lists, social media, etc.
== Day 1 (temporary) ==
 
<gallery>
File:IMG_20181012_161540990.jpg
File:IMG_20181012_161552147.jpg
File:IMG_20181012_161615376.jpg
File:IMG_20181012_161626209.jpg
File:IMG_20181012_161639871.jpg
File:IMG_20181012_162253725.jpg
</gallery>
 
= Raspberry Pi for the bookscanner =
 
At the moment we use an electromechanical button documented on the CHDK website that works well but does not do much more than triggering the two cameras to shoot at the same time: http://chdk.wikia.com/wiki/USB_Remote
 
There are many more possibilities of how to optimise the bookscanning process.  Many ideas start with connecting the cameras to a small computer such as the Raspberry Pi.  We made some experiments and tests with this setup, which is documented below:


= Raspberry Pi =
It runs [https://www.raspbian.org/ Raspbian].
It runs [https://www.raspbian.org/ Raspbian].


== CHDKPTP ==
== CHDKPTP ==
In /home/pi/chdkptp there are precompiled binaries of [https://app.assembla.com/spaces/chdkptp/wiki CHDKPTP] downloaded from [https://app.assembla.com/spaces/chdkptp/documents here].
In /home/pi/chdkptp there are precompiled binaries of [https://app.assembla.com/spaces/chdkptp/wiki CHDKPTP] downloaded from [https://app.assembla.com/spaces/chdkptp/documents here].


Line 226: Line 309:


Our setup has two modes of work:
Our setup has two modes of work:
* mechanical "button" which triggers both camera capturing its photo
* mechanical "button" which triggers both camera capturing its photo based on this tutorial: http://chdk.wikia.com/wiki/USB_Remote
* (in progress) Raspberry Pi mode where an bookscanner operator uses Raspberry Pi to capture photos, preview photos in real time and transfer them already renamed for the next step in Scantailor.
* (in progress) Raspberry Pi mode where an bookscanner operator uses Raspberry Pi to capture photos, preview photos in real time and transfer them already renamed for the next step in Scantailor.


Line 237: Line 320:
  <nowiki>sudo ./chdkptp.sh -e"connect -d=030" -erec -eshoot</nowiki>
  <nowiki>sudo ./chdkptp.sh -e"connect -d=030" -erec -eshoot</nowiki>
Above line connects to the camera sitting on -d=021, puts it into rec mode (if not already) and capture the photo saving it to SD card already in camera. If one wants to bypass the SD card altogether it should replace -eshoot with -eremoteshoot. In that case ./chdkptp.sh will save photos into the directory from where it was called.
Above line connects to the camera sitting on -d=021, puts it into rec mode (if not already) and capture the photo saving it to SD card already in camera. If one wants to bypass the SD card altogether it should replace -eshoot with -eremoteshoot. In that case ./chdkptp.sh will save photos into the directory from where it was called.
Note: check the SD card is locked.


== ZeroTier ==
== ZeroTier ==
It is added to 565799d8f6ebf1a8 public network of [https://zerotier.com ZeroTier] with this command:
It is added to 565799d8f6ebf1a8 public network of [https://zerotier.com ZeroTier] with this command:
  <nowiki>sudo zerotier-cli join 565799d8f6ebf1a8</nowiki>
  <nowiki>sudo zerotier-cli join 565799d8f6ebf1a8</nowiki>
Line 245: Line 331:


In /home/pi/.ssh/authorized_keys public keys of maxigas and marcell are added.
In /home/pi/.ssh/authorized_keys public keys of maxigas and marcell are added.
= Ideas for building Voja's scanner =
Our first idea was to reproduce Voja's scanner but we had to realise that we are not Voja and the scanner is very beautiful and unique engineering and consequently it is hard to reproduce.
Here are two links to the public documentation of our scanner, built by Voja Antonic:
https://www.memoryoftheworld.org/blog/2012/10/28/our-beloved-bookscanner-2/
https://hackaday.io/project/5604-diy-book-scanner
The electronics is not really documented (which means that it is hard to reproduce) and it is built from basic parts (which means that it takes a lot of time to put it together).  So we brainstorm about an Arduino-based solution instead.  Arduino is a general-purpose programmable microcontroller that has already built-in many of the functions/parts we need.  The idea is that this makes it easier for us to build the scanner and for others to reproduce it.  We also have more experience working with Arduino than with only basic electronic components.
We will also try to use cheaper cameras in order to bring down the budget.


= Biblio-graphy =
= Biblio-graphy =


== About our book scanner ==
== About our first bookscanner ==


[https://www.memoryoftheworld.org/blog/2012/10/28/our-beloved-bookscanner/ English]
[https://www.memoryoftheworld.org/blog/2012/10/28/our-beloved-bookscanner/ English]
Line 257: Line 357:


[http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]
[http://en.flossmanuals.net/e-book-enlightenment/ Reading And Leading With One Laptop Per Child]
== Future activities ==


[[next steps]]
[[next steps]]

Revision as of 23:21, 6 August 2019

Bookscanning in Calafou:

This page is documenting the design and construction of bookscanners. For the tutorial that explains "How to use the bookscanner", go to http://wiki.calafou.org/index.php/Bookscanning

Building a "New Standard Scanner"

We built this: https://forum.diybookscanner.org/viewtopic.php?f=1&t=333 during the Kunlabora event in Calafou: https://calafou.org/en/content/kunlabora-ephimeral-projects-kooperative A more reproducible and portable and simple scanner.

In theory for the simple scanner it should be possible to get all the ingredients from a hardware shop ("ferreteria"). We got most of the materials from Bauhaus in Barcelona (Zona Franca). Our experience was that most of the screws, nuts and bolts are easy to substitute with others that have more or less the same dimensions.

Bookscanner pseudoready2.jpg

Budget

Kunlabora event was four days of collaborative construction. More or less six persons worked on building the scanner. This is the receipts that we spent, because we had 500 EUR budget for the raw materials from the participation fee of the event.

Receipts:

   25    2x plexiglass [J] @ Expocryl, Igualada
   18    2x SD cards @ Life Informatica, Barcelona
   50    2x magic want camera stand @ FotoK, Barcelona
   42.2  Nuts/bolts/etc. @ Bauhaus, Zona Franca, Barcelona
   108.8 Nuts/bolts/etc. @ Bauhaus, Zona Franca, Barcelona
   10    2x USB cable male/female @ Worten San Antoni, Barcelona
   5     2x mini USB (USB-B) cable @ Tienda Cables, Barcelona
   178   2x Canon IXUS 175 compact digital camera @ Amazon.es
   60    4x Wooden beams (listones) @ Fustes Fargas, Barcelona
   ============================================
   497

Donations:

   70    Raspberry Pi 3 (Model B), 2x SD cards, 5V charger, rulers [M] @ Barcelona

We also used nuts and bolts, etc. found in the workshop of Calafou (in Catalunya, near Barcelona). The conclusion is that it should be possible to build this scanner from about 5-600 EUR anywhere in (Western) Europe. We did not use all the materials we bought and if we would do it again then we tried to use real wood instead of FDM, because the latter proved very fragile. However, this could make the platen more heavy, and it is already quite heavy.

Parts / componentes

01 SHOPPING 001.JPG

Going for shopping

Error creating thumbnail: File missing


From Bauhaus, Barcelona (Zona Franca):

01 SHOPPING 027.JPG
-1x Varilla roscada
-4x Listones de madera  ~800x40x90mm
-2x (Tablero) MD sheet 600x400x10mm
-2x (Tablero) MDF sheet 600x400x16mm
Error creating thumbnail: File missing
- Cola blanca rápida para madera
Error creating thumbnail: File missing
- Base 4 tomas 1.5 mts
01 SHOPPING 007.JPG
- Tuerca hexagonal
01 SHOPPING 020.JPG
- Guia corredera
01 SHOPPING 004.JPG
- Arandelas ancha
01 SHOPPING 005.JPG
- Aldabon BPF 185 M
01 SHOPPING 006.JPG
- Tornillo mad.cab.pl.
Error creating thumbnail: File missing
- Tornillo cab. avell.
01 SHOPPING 011.JPG
- Tornillo p/metal
01 SHOPPING 008.JPG
- Tuerca palomilla
01 SHOPPING 021.JPG
- Guia cajones 10kg
01 SHOPPING 022.JPG
- Spax cab. red zinc
01 SHOPPING 012.JPG
- Tirador dorado
01 SHOPPING 014.JPG
- 4x Abrazaderas
01 SHOPPING 025.JPG
- Mosqueton-fw
- 2 x plexi glass 380x280x3mm from a company in Igualada that J. found.

Other things from other shops:

01 SHOPPING 013.JPG
- 4x Cable USB 2.0 male-female @ Worten Sant Antoni
- 2x USB-A -> USB-B (mini) cables @ Tienda de Cables
- 2x SD Cards 16 Gg
Error creating thumbnail: File missing
- 2x Manfrotto brazo flexible MF237 (Nr. referencia AFP018036) @ FotoK, Ronda Universitat
01 SHOPPING 015.JPG
- 2x Tira LED 1 nice
- 2 x digital cameras (see below for background information): Canon IXUS 175 (Powershot / ELPH 180)
  - https://www.amazon.es/Canon-IXUS-175-compacta-estabilizador/dp/B01A8QU70I/
  - http://chdk.wikia.com/wiki/ELPH180
  - https://www.canon-europe.com/for_home/product_finder/cameras/digital_camera/ixus/ixus_175/specification.aspx

For electronic parts a good shop in Barcelona is Diotronic: https://diotronic.com/

- usb charger to give power to the external trigger trough USB cable

Things to take in consideration from the building experience

  • the MDF pieces are fragile and gave us problems when introducing screws
  • we considered varnishing the wood pieces to limit humidity impact
  • the plexiglas we bought generates many reflections
  • passing from inches to cm was uncomfortable
  • the position and height of the light should be tested before being fixed
  • the upper part of the light stand should be easier to mount and unmount for transportation
  • the wood used for the platen should be without texture or painted with mate dark paint
  • the gooseneck (Manfrotto) option for the camera stand is not a good option: it is very easy to put the camera in the way of the cradle; the calibration process is difficult because the cameras can move in the three axis; they are springy and difficult to set in an exact position

Cameras

The bookscanner is basically a glorified camera stand that allows us to take good pictures of book pages. So the cameras are an important part of the bookscanner. Summary of research about cameras for book scanners:

Basically there are three categories of cameras that can be used for book scanners (from cheapest to most expensive).

1. Remote control support

The cheapest option is any camera with remote trigger support, so we can take pictures without pushing the button on the camera. This is important because when you press the button the camera position may be disadjusted to the physical pressure.

2. CHDK firmware

Middle category is CHDK firmware compatibles. CHDK is a third party open source firmware that allows the customisation of cameras. CHDK firmware is for Canon Powershot cameras, which are the cheaper compact digital camera product line. We had 200 euros in the budget for cameras, so we went with this option.

3. Magic Lantern support

Magic Lantern is a third party open source firmware that is more advanced. However, it only works with Canon DLSR cameras (these are the cameras that have a reflex mirror to look at the shot through a small hole before you take the picture, and they usually have big lenses). The scanner we have now uses Canon 1100D, which are the cheapest type suported by Magic Lantern, but they still cost a few hundred euros.

4. Connecting to a monitor

It is very useful to be able to connect the cameras to a monitor during the calibration process. The cameras we bought use the same output for connecting to a remote control and to a monitor making it not practical at all using a monitor. Furthermore, connection with usb to a computer monitor was not an option, we had to look for RC connectors or adapters.

5. A note about webcams:

An option might seem to use two high resolution webcams directly connected to a laptop to trigger taking pictures and monitor their quality, and there are full HD webcams (~2 Megapixels). The problem with all webcams, however, is that they have a fixed optical zoom, at quite a wide angle. To get those pixels covering the book page, you need to be very close to the book (~25cm). This forces you to use a design similar to what Reece Arnott uses, or the Ion Audio Booksaver. These designs make it impossible to avoid glare from light sources, no matter where the lights are placed. To avoid glare, you need a narrow zoom and a good distance (~90 cm) between book and camera.

6. Follow up: document cameras

The digital cameras mentioned above proved unreliable on the long run, because it was difficult/impossible to make them focus on the page. It seems this is a problem with this particular model, reported by many users on forums. The solution was to invest more money and buy specialised "document cameras", such as this one: <https://www.ipevo.com/product/v4k/gallery>

Building process

Base / Base

02 BASE 035.JPG
02 BASE 043.JPG
02 BASE 040.JPG

We cut the wood beams to have 2 pieces of 56x4cm and 2 of 31.5x4cm

02 BASE 044.JPG
02 BASE 041.JPG

we measured with precision tools and marked the position of the drawer sliders on the 54x4cm wood beams

02 BASE 045.JPG

we drilled holes on the marked position

Error creating thumbnail: File missing
Error creating thumbnail: File missing
02 BASE 051.JPG

and then we fixed the sliders

02 BASE 033.JPG

We used the thicker MDF material to cut a 37x29cm sheet

02 BASE 056.JPG

The last parts were fixing the other part of the sliders to the MDF sheet and attaching the different part of the base with screws


Cradle / Cuna

The cradle is a V-shaped book holder partly fixed to the base

07 GEN 036.JPG

We cut two MDF (thin) sheets for the book holder

03 CRADDLE 037.JPG
03 CRADDLE 046.JPG

And 4 small triangles to maintain the book holder on the base with a 90 degree angle between both sheets

03 CRADDLE 057.JPG

We measured were to position the triangles

Error creating thumbnail: File missing

Drilled the MDF sheet on the base

03 CRADDLE 058.JPG

And fixed the MDF small triangles

03 CRADDLE 059.JPG

We drilled the MDF sheet of the Craddle

Error creating thumbnail: File missing

And fixed it to the triangles

NOT DOCUMENTED WITH PICTURES

05 COLUMN 066.JPG

The 2 other triangles were fixed to the other MDF sheet without being fixed to the base. The distance between the triangles is of 19.5cm. We cut a 19.5x19.5cm MDF square that we attached to the other side of the base. The drilled the 2 triangles in order to introduce a threaded rod. The threaded rod is used to tighten the triangles against the square fixed on the base.


Platen / Platina

Error creating thumbnail: File missing
04 PLATEN 047.JPG
04 PLATEN 053.JPG
04 PLATEN 054.JPG
04 PLATEN 055.JPG
Error creating thumbnail: File missing


Column / Columna

Error creating thumbnail: File missing
05 COLUMN 066.JPG
05 COLUMN 067.JPG
05 COLUMN 068.JPG
Error creating thumbnail: File missing
Error creating thumbnail: File missing
05 COLUMN 073.JPG
05 COLUMN 074.JPG
05 COLUMN 075.JPG
05 COLUMN 076.JPG
Error creating thumbnail: File missing
05 COLUMN 078.JPG
05 COLUMN 079.JPG
05 COLUMN 080.JPG
Error creating thumbnail: File missing
05 COLUMN 084.JPG
06 MOUNTED 086.JPG

CHDK

External trigger

08 TRIGERING 081.JPG
08 TRIGERING 082.JPG

Photos

Mounting

Miscellaneous

Triggering mechanism

Camera holder

Day 1 (temporary)

Raspberry Pi for the bookscanner

At the moment we use an electromechanical button documented on the CHDK website that works well but does not do much more than triggering the two cameras to shoot at the same time: http://chdk.wikia.com/wiki/USB_Remote

There are many more possibilities of how to optimise the bookscanning process. Many ideas start with connecting the cameras to a small computer such as the Raspberry Pi. We made some experiments and tests with this setup, which is documented below:

It runs Raspbian.

CHDKPTP

In /home/pi/chdkptp there are precompiled binaries of CHDKPTP downloaded from here.

CHDKPTP is used for remote control of camera running CHDK firmware.

Our setup has two modes of work:

  • mechanical "button" which triggers both camera capturing its photo based on this tutorial: http://chdk.wikia.com/wiki/USB_Remote
  • (in progress) Raspberry Pi mode where an bookscanner operator uses Raspberry Pi to capture photos, preview photos in real time and transfer them already renamed for the next step in Scantailor.

When camera is connected this line will list info about the camera e.g.:

sudo ./chdkptp.sh -elist

and for one of the camera this is what is listed then:

-1:Canon IXUS 175 b=001 d=030 v=0x4a9 p=0x32c1 s=8B20D62641B041BAA3E1D597D560D110

An example of capturing a picture from commandline (once in /home/pi/chdkptp/):

sudo ./chdkptp.sh -e"connect -d=030" -erec -eshoot

Above line connects to the camera sitting on -d=021, puts it into rec mode (if not already) and capture the photo saving it to SD card already in camera. If one wants to bypass the SD card altogether it should replace -eshoot with -eremoteshoot. In that case ./chdkptp.sh will save photos into the directory from where it was called.

Note: check the SD card is locked.

ZeroTier

It is added to 565799d8f6ebf1a8 public network of ZeroTier with this command:

sudo zerotier-cli join 565799d8f6ebf1a8

and it got static IP address (in 565799d8f6ebf1a8 network):

192.168.192.171/24

In /home/pi/.ssh/authorized_keys public keys of maxigas and marcell are added.

Ideas for building Voja's scanner

Our first idea was to reproduce Voja's scanner but we had to realise that we are not Voja and the scanner is very beautiful and unique engineering and consequently it is hard to reproduce.

Here are two links to the public documentation of our scanner, built by Voja Antonic:

https://www.memoryoftheworld.org/blog/2012/10/28/our-beloved-bookscanner-2/

https://hackaday.io/project/5604-diy-book-scanner

The electronics is not really documented (which means that it is hard to reproduce) and it is built from basic parts (which means that it takes a lot of time to put it together). So we brainstorm about an Arduino-based solution instead. Arduino is a general-purpose programmable microcontroller that has already built-in many of the functions/parts we need. The idea is that this makes it easier for us to build the scanner and for others to reproduce it. We also have more experience working with Arduino than with only basic electronic components.

We will also try to use cheaper cameras in order to bring down the budget.

Biblio-graphy

About our first bookscanner

English

Spanish

Principal sources

Reading And Leading With One Laptop Per Child

Future activities

next steps