avans06 commited on
Commit
8135b6a
·
1 Parent(s): 36585bd

init commit.

Browse files

# Kumiko Manga/Comics Panel Extractor (WebUI)

Upload your manga or comic book images. This tool will automatically analyze the panels on each page,
crop them into individual image files, and package them into a single ZIP file for you to download.

This application is licensed under the **GNU Affero General Public License v3.0**.
The core panel detection is powered by the **kumikolib** library, created by **njean42** ([Original Project](https://github.com/njean42/kumiko)).

Files changed (12) hide show
  1. .gitignore +7 -0
  2. LICENSE +18 -0
  3. README.md +127 -4
  4. app.py +243 -0
  5. kumikolib.py +130 -0
  6. lib/debug.py +280 -0
  7. lib/html.py +106 -0
  8. lib/page.py +404 -0
  9. lib/panel.py +479 -0
  10. lib/segment.py +197 -0
  11. requirements.txt +3 -0
  12. webui.bat +162 -0
.gitignore ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ __pycache__
2
+ *.pyc
3
+ tests/*.html
4
+ tests/results/*
5
+ .vs/
6
+ venv/
7
+ tmp/
LICENSE ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+
3
+ Kumiko, the Comics Cutter is a set of tool for generating information about
4
+ panels locations within a comic books pages, and more.
5
+ Copyright (C) 2018 njean42
6
+
7
+ This program is free software: you can redistribute it and/or modify
8
+ it under the terms of the GNU Affero General Public License as
9
+ published by the Free Software Foundation, either version 3 of the
10
+ License, or (at your option) any later version.
11
+
12
+ This program is distributed in the hope that it will be useful,
13
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
14
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15
+ GNU Affero General Public License for more details.
16
+
17
+ You should have received a copy of the GNU Affero General Public License
18
+ along with this program. If not, see <https://www.gnu.org/licenses/>.
README.md CHANGED
@@ -1,13 +1,136 @@
1
  ---
2
  title: KumikoMangaPanelExtractor
3
- emoji: 😻
4
- colorFrom: gray
5
- colorTo: pink
6
  sdk: gradio
7
  sdk_version: 5.35.0
8
  app_file: app.py
9
  pinned: false
10
- short_description: Kumiko Manga/Comics Pane lExtractor (WebUI)
11
  ---
12
 
13
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: KumikoMangaPanelExtractor
3
+ emoji: 🕮
4
+ colorFrom: indigo
5
+ colorTo: green
6
  sdk: gradio
7
  sdk_version: 5.35.0
8
  app_file: app.py
9
  pinned: false
10
+ short_description: Kumiko Manga/Comics Panel Extractor (WebUI)
11
  ---
12
 
13
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
14
+
15
+ ___
16
+
17
+
18
+ # Kumiko Manga/Comics Panel Extractor (WebUI)
19
+
20
+ Upload your manga or comic book images. This tool will automatically analyze the panels on each page,
21
+ crop them into individual image files, and package them into a single ZIP file for you to download.
22
+
23
+ This application is licensed under the **GNU Affero General Public License v3.0**.
24
+ The core panel detection is powered by the **kumikolib** library, created by **njean42** ([Original Project](https://github.com/njean42/kumiko)).
25
+
26
+ ___
27
+
28
+
29
+ # Introduction
30
+
31
+ ![Kumiko mascot by Cthulhulumaid](https://raw.githubusercontent.com/njean42/kumiko/refs/heads/master/artwork/kumiko-big.png "Kumiko mascot by Cthulhulumaid")
32
+
33
+ > Kumiko mascot by [Hurluberlue](https://www.twitch.tv/hurluberlue "twitch link"), [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/ "Creative Commons License")
34
+
35
+ *Kumiko, the Comics Cutter* is a set of tools to compute useful information about comic book pages, panels, and more.
36
+ Its main strength is to find out the **locations of panels** within a comic's page (image file).
37
+ *Kumiko* can also compile information about panels for all pages in a comic book, and present it as one piece of data (JSON-formatted object).
38
+
39
+ *Kumiko* makes use of the great (freely licensed) [opencv](https://opencv.org/) library, which provides image processing algorithms of all sorts.
40
+ Mainly, the contour detection algorithm is used to detect panels within an image.
41
+
42
+
43
+ # Demo
44
+
45
+ *TL;WR* Too Long; Won't Read the whole doc?
46
+
47
+ A **live demo** is [available here](https://kumiko.njean.me/demo), where you can try *Kumiko* out and cut your own comic pages into panels.
48
+
49
+
50
+ # Philosophy
51
+
52
+ *Kumiko* aims at being a functional library to extract information from comic pages / books.
53
+ The goal is to provide a set of tools that is usable beforehand, to extract all needed information.
54
+
55
+ External programs can later use the generated information for different purposes: panel-by-panel viewing, actual splitting of an image down into panels, etc.
56
+
57
+
58
+ ## Panel-by-panel comic reading
59
+
60
+ Being able to jump from one panel to the next was the original idea behind *Kumiko*.
61
+
62
+ ![xkcd #208](doc/img/xkcd.png "xkcd #208")
63
+
64
+ > [xkcd](https://www.xkcd.com) by Randall Munroe, [#208](https://www.xkcd.com/208/), [CC BY-NC 2.5](https://creativecommons.org/licenses/by-nc/2.5/)
65
+
66
+ Comic viewers usually imply a very common *page-by-page reading paradigm*.
67
+ You read a page, possibly zooming on it to be able to read speech bubbles, then click, tap, press a key or swipe to the next page.
68
+
69
+ With knowledge about panels locations, we can imagine a comic reader that also offers *panel-by-panel reading*.
70
+ This is especially interesting for **small screens**, on which you probably can't read the texts if a whole page is displayed.
71
+
72
+ Just run `kumiko -i /path/to/comicpage.jpg -b firefox` on your *comicpage.jpg* file, and read it panel-by-panel in your browser!
73
+
74
+
75
+ # Requirements
76
+
77
+ `apt-get install python3-opencv` will install the only necessary library needed: *opencv*.
78
+
79
+ This should do the trick for Debian distros and derivatives (Ubuntu, Linux Mint...).
80
+ If you successfully use *Kumiko* on any other platform, please let us know!
81
+
82
+
83
+ # Usage & Testing
84
+
85
+ See the [usage doc](doc/Usage.md) for details on how to use the *Kumiko* tools.
86
+
87
+ Also check the [testing doc](doc/Testing.md) if you want to test modified versions of the code.
88
+
89
+
90
+ # Numbering
91
+
92
+ The numbering is left-to-right, or right-to-left if requested.
93
+
94
+ Here is an example of how *Kumiko* is going to number panels by default (numbers and red lines not in the original picture).
95
+
96
+ ![Pepper&Carrot](doc/img/numbering.png "Pepper&Carrot")
97
+
98
+ > [Pepper & Carott](https://www.peppercarrot.com/) by [David Revoy](https://www.davidrevoy.com), [episode 2](https://www.peppercarrot.com/en/article237/episode-2-rainbow-potions), [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
99
+
100
+
101
+ # Contributing
102
+
103
+ Feature requests and PR are welcome!
104
+
105
+ *Kumiko* python code if formatted with [yapf](https://github.com/google/yapf).
106
+ Config file is committed [here](.style.yapf).
107
+
108
+ To format all your code, simply run:
109
+ ```bash
110
+ yapf3 --recursive --in-place .
111
+ ```
112
+
113
+
114
+ # Short- and longer-term features (roadmap)
115
+
116
+ ## Kumiko library
117
+
118
+ * detect panels on a growing range of comic page layouts
119
+ * detect non-framed panels (without clear boundaries/borders)
120
+ * separate intertwined panels
121
+
122
+ * ~~be able to detect panel contours on pages with non-white, non-black background~~ done in v1.5
123
+
124
+ ## Back-office (validation / edition tool)
125
+
126
+ Let's face it: we probably can't ensure that *Kumiko* can perfectly find out the panels in *any* image.
127
+ There is a huge diversity of panel boundaries, layouts and whatnot.
128
+
129
+ This is why there could be some kind of back-office / editing tool that lets a human editor:
130
+
131
+ * validate pages
132
+ * add, delete, move or resize incorrect panels
133
+ * report bugs
134
+ * ...
135
+
136
+ Such a tool would edit the JSON file representing a comic book information, for later use by other programs relying on it.
app.py ADDED
@@ -0,0 +1,243 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Kumiko Manga/Comics Panel Extractor (WebUI)
3
+ Copyright (C) 2025 avan
4
+
5
+ This program is a web interface for the Kumiko library.
6
+ The core logic is based on Kumiko, the Comics Cutter.
7
+ Copyright (C) 2018 njean42
8
+
9
+ This program is free software: you can redistribute it and/or modify
10
+ it under the terms of the GNU Affero General Public License as
11
+ published by the Free Software Foundation, either version 3 of the
12
+ License, or (at your option) any later version.
13
+
14
+ This program is distributed in the hope that it will be useful,
15
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
16
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17
+ GNU Affero General Public License for more details.
18
+
19
+ You should have received a copy of the GNU Affero General Public License
20
+ along with this program. If not, see <https://www.gnu.org/licenses/>.
21
+ """
22
+
23
+ import gradio as gr
24
+ import os
25
+ import tempfile
26
+ import shutil
27
+ import numpy as np
28
+ import cv2 as cv
29
+
30
+ # Import Kumiko's core library and its page module dependency
31
+ import kumikolib
32
+ import lib.page
33
+
34
+ # ----------------------------------------------------------------------
35
+ # Core functions to solve the non-English path issue
36
+ # ----------------------------------------------------------------------
37
+
38
+ def imread_unicode(filename, flags=cv.IMREAD_COLOR):
39
+ """
40
+ Replaces cv.imread to support non-ASCII paths.
41
+ """
42
+ try:
43
+ with open(filename, 'rb') as f:
44
+ n = np.frombuffer(f.read(), np.uint8)
45
+ img = cv.imdecode(n, flags)
46
+ return img
47
+ except Exception as e:
48
+ print(f"Error reading file {filename}: {e}")
49
+ return None
50
+
51
+ def imwrite_unicode(filename, img):
52
+ """
53
+ Replaces cv.imwrite to support non-ASCII paths.
54
+ """
55
+ try:
56
+ ext = os.path.splitext(filename)[1]
57
+ if not ext:
58
+ ext = ".jpg" # Default to jpg if no extension
59
+ result, n = cv.imencode(ext, img)
60
+ if result:
61
+ with open(filename, 'wb') as f:
62
+ f.write(n)
63
+ return True
64
+ else:
65
+ return False
66
+ except Exception as e:
67
+ print(f"Error writing to file {filename}: {e}")
68
+ return False
69
+
70
+ # ----------------------------------------------------------------------
71
+ # Monkey Patching
72
+ # This dynamically replaces the problematic functions in the original
73
+ # libraries without modifying their source code files.
74
+ # ----------------------------------------------------------------------
75
+
76
+ # Replace the cv.imread used in page.py with our new version
77
+ lib.page.cv.imread = imread_unicode
78
+
79
+ # Replace the cv.imwrite used in kumikolib.py with our new version
80
+ kumikolib.cv.imwrite = imwrite_unicode
81
+
82
+
83
+ # ----------------------------------------------------------------------
84
+ # Gradio Processing Function
85
+ # ----------------------------------------------------------------------
86
+
87
+ def process_manga_images(files, output_structure, use_rtl, progress=gr.Progress(track_tqdm=True)):
88
+ """
89
+ The main processing logic for the Gradio interface.
90
+ Receives uploaded files and settings, processes them, and returns a path to a ZIP file.
91
+ """
92
+ if not files:
93
+ raise gr.Error("Please upload at least one image file.")
94
+
95
+ # Create temporary directories for processing
96
+ # 1. To store the cropped panel images
97
+ # 2. To store the final ZIP archive
98
+ panel_output_dir = tempfile.mkdtemp(prefix="kumiko_panels_")
99
+ zip_output_dir = tempfile.mkdtemp(prefix="kumiko_zip_")
100
+
101
+ try:
102
+ # The 'files' object from gr.Files is a list of temporary file objects
103
+ image_paths = [file.name for file in files]
104
+
105
+ progress(0, desc="Initializing Kumiko...")
106
+
107
+ # Initialize Kumiko with the rtl setting from the UI
108
+ k = kumikolib.Kumiko({
109
+ 'debug': False,
110
+ 'progress': False, # We use Gradio's progress bar instead
111
+ 'rtl': use_rtl, # Use the value from the checkbox
112
+ 'panel_expansion': True,
113
+ })
114
+
115
+ # 1. Analyze all images
116
+ total_files = len(image_paths)
117
+ for i, path in enumerate(image_paths):
118
+ progress((i + 1) / total_files, desc=f"Analyzing: {os.path.basename(path)}")
119
+ try:
120
+ k.parse_image(path)
121
+ except lib.page.NotAnImageException as e:
122
+ print(f"Warning: Skipping file {os.path.basename(path)} because it is not a valid image. Error: {e}")
123
+ continue
124
+
125
+ # 2. Save the panels based on the selected output structure
126
+ # This section replaces the original `k.save_panels()` call.
127
+ progress(0.8, desc="Saving all panels...")
128
+ nb_written_panels = 0
129
+ for page in k.page_list:
130
+ original_filename_base = os.path.splitext(os.path.basename(page.filename))[0]
131
+
132
+ for i, panel in enumerate(page.panels):
133
+ x, y, width, height = panel.to_xywh()
134
+ panel_img = page.img[y:y + height, x:x + width]
135
+
136
+ output_filepath = ""
137
+ # Check user's choice for the output structure
138
+ if output_structure == "Group panels in folders":
139
+ # Default behavior: one folder per image
140
+ image_specific_dir = os.path.join(panel_output_dir, original_filename_base)
141
+ os.makedirs(image_specific_dir, exist_ok=True)
142
+ output_filename = f"panel_{i}.jpg"
143
+ output_filepath = os.path.join(image_specific_dir, output_filename)
144
+ else: # "Create a flat directory"
145
+ # New behavior: flat structure with prefixed filenames
146
+ output_filename = f"{original_filename_base}_panel_{i}.jpg"
147
+ output_filepath = os.path.join(panel_output_dir, output_filename)
148
+
149
+ # Save the panel using our unicode-safe writer
150
+ if imwrite_unicode(output_filepath, panel_img):
151
+ nb_written_panels += 1
152
+ else:
153
+ print(f"\n[ERROR] Failed to write panel image to {output_filepath}\n")
154
+
155
+ # 3. Package all cropped panels into a ZIP file
156
+ progress(0.9, desc="Creating ZIP archive...")
157
+ if nb_written_panels == 0:
158
+ raise gr.Error("Analysis complete, but no croppable panels were detected.")
159
+
160
+ zip_filename_base = os.path.join(zip_output_dir, "kumiko_output")
161
+ zip_filepath = shutil.make_archive(zip_filename_base, 'zip', panel_output_dir)
162
+
163
+ progress(1, desc="Done!")
164
+
165
+ return zip_filepath
166
+
167
+ except Exception as e:
168
+ # Catch any other potential errors during processing
169
+ raise gr.Error(f"An error occurred during processing: {e}")
170
+ finally:
171
+ # 4. Clean up the temporary directory for panels, regardless of success or failure
172
+ shutil.rmtree(panel_output_dir)
173
+ # Note: Gradio will automatically handle the cleanup of zip_output_dir because a file from it is returned.
174
+
175
+ # ----------------------------------------------------------------------
176
+ # Create and Launch the Gradio Interface
177
+ # ----------------------------------------------------------------------
178
+
179
+ with gr.Blocks(theme=gr.themes.Soft()) as demo:
180
+ gr.Markdown(
181
+ """
182
+ # Kumiko Manga/Comics Panel Extractor (WebUI)
183
+ Upload your manga or comic book images. This tool will automatically analyze the panels on each page,
184
+ crop them into individual image files, and package them into a single ZIP file for you to download.
185
+
186
+ This application is licensed under the **GNU Affero General Public License v3.0**.
187
+ The core panel detection is powered by the **kumikolib** library, created by **njean42** ([Original Project](https://github.com/njean42/kumiko)).
188
+ """
189
+ )
190
+
191
+ with gr.Row():
192
+ with gr.Column(scale=1):
193
+ image_input = gr.Files(
194
+ label="Upload Manga Images",
195
+ file_count="multiple",
196
+ file_types=["image"],
197
+ )
198
+
199
+ with gr.Accordion("Advanced Settings", open=False):
200
+ # Add the new Radio button for selecting the output structure
201
+ output_structure_choice = gr.Radio(
202
+ label="ZIP File Structure",
203
+ choices=["Group panels in folders", "Create a flat directory"],
204
+ value="Group panels in folders", # Default value
205
+ info="Choose how to organize panels in the output ZIP file."
206
+ )
207
+
208
+ # Add the new Checkbox for RTL setting
209
+ rtl_checkbox = gr.Checkbox(
210
+ label="Right-to-Left (RTL) Reading Order",
211
+ value=False, # Default to False
212
+ info="Check this for manga that is read from right to left."
213
+ )
214
+
215
+ process_button = gr.Button("Start Analysis & Cropping", variant="primary")
216
+
217
+ with gr.Column(scale=1):
218
+ output_zip = gr.File(
219
+ label="Download Cropped Panels (ZIP)",
220
+ )
221
+
222
+ process_button.click(
223
+ fn=process_manga_images,
224
+ inputs=[image_input, output_structure_choice, rtl_checkbox],
225
+ outputs=output_zip,
226
+ api_name="process"
227
+ )
228
+
229
+ # gr.Examples(
230
+ # examples=[
231
+ # [
232
+ # [os.path.join(os.path.dirname(__file__), "example1.jpg"), os.path.join(os.path.dirname(__file__), "example2.png")]
233
+ # ]
234
+ # ],
235
+ # inputs=image_input,
236
+ # outputs=output_zip,
237
+ # fn=process_manga_images,
238
+ # label="Examples (place example1.jpg and example2.png in the same directory as this script)"
239
+ # )
240
+
241
+
242
+ if __name__ == "__main__":
243
+ demo.launch(inbrowser=True)
kumikolib.py ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import sys
3
+ import tempfile
4
+ import cv2 as cv
5
+ import numpy as np
6
+ import requests
7
+ import subprocess
8
+ from urllib.parse import urlparse
9
+
10
+ from lib.page import Page, NotAnImageException
11
+ from lib.debug import Debug
12
+
13
+
14
+ class Kumiko:
15
+
16
+ options = {}
17
+
18
+ def __init__(self, options = None):
19
+ options = options or {}
20
+
21
+ for o in ['progress', 'rtl', 'debug']:
22
+ self.options[o] = o in options and options[o]
23
+
24
+ Debug.debug = self.options['debug']
25
+
26
+ self.options['min_panel_size_ratio'] = options.get('min_panel_size_ratio', None)
27
+
28
+ self.panel_expansion = options.get('panel_expansion', True)
29
+
30
+ self.page_list = []
31
+
32
+ def parse_url_list(self, urls):
33
+ if self.options['progress']:
34
+ print(len(urls), 'files to download', file = sys.stderr)
35
+
36
+ self.temp_folder = tempfile.TemporaryDirectory()
37
+
38
+ i = 0
39
+ nbdigits = len(str(len(urls)))
40
+ for url in urls:
41
+ filename = 'img' + ('0' * nbdigits + str(i))[-nbdigits:]
42
+
43
+ if self.options['progress']:
44
+ print('\t', url, (' -> ' + filename) if urls else '', file = sys.stderr)
45
+
46
+ i += 1
47
+ parts = urlparse(url)
48
+ if not parts.netloc or not parts.path:
49
+ continue
50
+
51
+ r = requests.get(url, timeout = 5)
52
+ with open(os.path.join(self.temp_folder.name, filename), 'wb') as f:
53
+ f.write(r.content)
54
+
55
+ self.parse_dir(self.temp_folder.name, urls = urls)
56
+
57
+ def parse_pdf_file(self, pdf_filename):
58
+ try:
59
+ subprocess.run(args = ['pdftoppm', '--help'], check = True, capture_output = True)
60
+ except FileNotFoundError:
61
+ print("Please `apt install pdftoppm` if you give PDF --input files to Kumiko", file = sys.stderr)
62
+ sys.exit(1)
63
+
64
+ self.temp_folder = tempfile.mkdtemp(prefix = "kumiko-pdf-pages-")
65
+
66
+ print(f"Using pdftoppm to extract jpeg files from pdf to {self.temp_folder}", file = sys.stderr)
67
+ subprocess.run(args = ['pdftoppm', '-jpeg', pdf_filename, f"{self.temp_folder}/"], check = True)
68
+
69
+ self.parse_dir(self.temp_folder)
70
+
71
+ def parse_dir(self, directory, urls = None):
72
+ filenames = []
73
+ for filename in os.scandir(directory):
74
+ filenames.append(filename.path)
75
+ self.parse_images(filenames, urls)
76
+
77
+ def parse_images(self, filenames, urls = None):
78
+ if self.options['progress']:
79
+ print(len(filenames), 'files to cut panels for', file = sys.stderr)
80
+
81
+ i = -1
82
+ for filename in sorted(filenames):
83
+ i += 1
84
+ if self.options['progress']:
85
+ print("\t", urls[i] if urls else filename, file = sys.stderr)
86
+
87
+ try:
88
+ self.parse_image(filename, url = urls[i] if urls else None)
89
+ except NotAnImageException:
90
+ if not filename.endswith(".license"):
91
+ print(f"\n[ERROR] Not an image, will be ignored: {filename}\n", file = sys.stderr)
92
+
93
+ def parse_image(self, filename, url = None):
94
+ self.page_list.append(
95
+ Page(
96
+ filename,
97
+ numbering = "rtl" if self.options['rtl'] else "ltr",
98
+ url = url,
99
+ min_panel_size_ratio = self.options['min_panel_size_ratio'],
100
+ panel_expansion = self.panel_expansion,
101
+ )
102
+ )
103
+
104
+ def get_infos(self):
105
+ return list(map(lambda p: p.get_infos(), self.page_list))
106
+
107
+ def save_panels(self, output_base_path = 'auto', output_format = "jpg"):
108
+ if output_base_path == 'auto':
109
+ output_base_path = tempfile.mkdtemp(prefix = "kumiko-out-")
110
+ elif not os.path.isdir(output_base_path):
111
+ print(
112
+ f"\n[ERROR] Given --save-panels directory is not a directory: {output_base_path}\n", file = sys.stderr
113
+ )
114
+ sys.exit(1)
115
+
116
+ nb_written_panels = 0
117
+ for page in self.page_list:
118
+ output_path = os.path.join(output_base_path, os.path.basename(page.filename))
119
+ os.makedirs(output_path, exist_ok = True)
120
+
121
+ for i, panel in enumerate(page.panels):
122
+ x, y, width, height = panel.to_xywh()
123
+ output_file = os.path.join(output_path, f"panel_{i}.{output_format}")
124
+ panel = page.img[y:y + height, x:x + width]
125
+ if cv.imwrite(output_file, panel):
126
+ nb_written_panels += 1
127
+ else:
128
+ print(f"\n[ERROR] Failed to write panel image to {output_file}\n", file = sys.stderr)
129
+
130
+ print(f"Saved {nb_written_panels} panel images to {output_base_path}", file = sys.stderr)
lib/debug.py ADDED
@@ -0,0 +1,280 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import json
3
+ import copy
4
+ import re
5
+ import time
6
+ import cv2 as cv
7
+ import numpy as np
8
+
9
+ from lib.html import HTML
10
+
11
+
12
+ class Debug:
13
+
14
+ colours = {
15
+ 'white': (255, 255, 255),
16
+ 'red': (0, 0, 255),
17
+ 'green': (0, 255, 0),
18
+ 'blue': (255, 0, 0),
19
+ 'lightblue': (200, 200, 0),
20
+ 'lightpurple': (200, 0, 200),
21
+ 'yellow': (0, 200, 200),
22
+ 'gray': (150, 150, 150),
23
+ }
24
+
25
+ # white, red and green are used to display main panels
26
+ subpanel_colours = list(colours.values())[3:]
27
+
28
+ debug = False
29
+ contour_size = None
30
+ steps = []
31
+ images = {}
32
+ time = time.time_ns()
33
+ base_img = img = None
34
+
35
+ @staticmethod
36
+ def set_base_img(img):
37
+ if not Debug.debug:
38
+ return
39
+
40
+ Debug.base_img = img
41
+ Debug.img = np.copy(img)
42
+
43
+ @staticmethod
44
+ def add_step(name, infos):
45
+ if not Debug.debug:
46
+ return
47
+
48
+ elapsed = Debug.show_time(f"{name} ({len(infos['panels'])} panels)")
49
+
50
+ Debug.steps.append({
51
+ 'name': name,
52
+ 'elapsed_since_last_step': elapsed,
53
+ 'infos': copy.deepcopy(infos),
54
+ })
55
+
56
+ @staticmethod
57
+ def show_time(name):
58
+ if not Debug.debug:
59
+ return
60
+
61
+ Debug.prev_time = Debug.time
62
+ Debug.time = time.time_ns()
63
+
64
+ elapsed = Debug.time - Debug.prev_time
65
+ print(f"{name} − {elapsed/pow(10,6):.0f}ms")
66
+
67
+ return elapsed
68
+
69
+ imgID = 0
70
+
71
+ @staticmethod
72
+ def add_image(label, img = None):
73
+ if not Debug.debug:
74
+ return
75
+
76
+ clean_filename = re.sub(r'\W', '-', label)
77
+ filename = f"{Debug.imgID}-{clean_filename}.jpg"
78
+ Debug.imgID += 1
79
+ cv.imwrite(os.path.join('tests/results', filename), Debug.img if img is None else img)
80
+
81
+ # reinit image so we see only specific steps' contours/lines/dots
82
+ Debug.img = np.copy(Debug.base_img)
83
+
84
+ currstep = len(Debug.steps) - 1
85
+ if currstep not in Debug.images:
86
+ Debug.images[currstep] = []
87
+ Debug.images[currstep].append({'filename': filename, 'label': label})
88
+
89
+ @staticmethod
90
+ def html(images_dir, reldir):
91
+ html = ''
92
+ html += HTML.header(title = 'Debugging - Kumiko processing steps', reldir = reldir)
93
+
94
+ for i in range(len(Debug.steps) - 1):
95
+ j = i + 1
96
+
97
+ # Display debug images
98
+ if i in Debug.images:
99
+ html += HTML.imgbox(Debug.images[i])
100
+
101
+ # Display panels diffs
102
+ files_diff = Debug.get_files_diff(images_dir, [Debug.steps[i]['infos']], [Debug.steps[j]['infos']])
103
+
104
+ step_name = str(i + 1) + '. ' + Debug.steps[j]['name']
105
+
106
+ if len(files_diff) == 0:
107
+ html += f"<h2>{step_name} - no change</h2>"
108
+
109
+ for _, diffs in files_diff.items():
110
+ html += HTML.side_by_side_panels(
111
+ step_name,
112
+ f"took {Debug.steps[j]['elapsed_since_last_step']/pow(10,9):.2f} seconds",
113
+ diffs['jsons'],
114
+ f"BEFORE - {len(diffs['jsons'][0][0]['panels'])} panels",
115
+ f"AFTER - {len(diffs['jsons'][1][0]['panels'])} panels",
116
+ images_dir = diffs['images_dir'],
117
+ known_panels = diffs['known_panels'],
118
+ diff_numbering_panels = diffs['diff_numbering_panels'],
119
+ )
120
+
121
+ html += HTML.footer
122
+ return html
123
+
124
+ @staticmethod
125
+ def get_files_diff(file_or_dir, json1, json2):
126
+ from lib.panel import Panel
127
+
128
+ files_diff = {}
129
+
130
+ for p in range(len(json1)): # for each page
131
+
132
+ # check both images' filename and size, should be the same
133
+ if os.path.basename(json1[p]['filename']) != os.path.basename(json2[p]['filename']):
134
+ print('error, filenames are not the same', json1[p]['filename'], json2[p]['filename'])
135
+ continue
136
+ if json1[p]['size'] != json2[p]['size']:
137
+ print('error, image sizes are not the same', json1[p]['size'], json2[p]['size'])
138
+ continue
139
+
140
+ panels_v1 = list(map(lambda p: Panel(None, p), json1[p]['panels']))
141
+ panels_v2 = list(map(lambda p: Panel(None, p), json2[p]['panels']))
142
+
143
+ known_panels = [[], []]
144
+ j = -1
145
+ for p1 in panels_v1:
146
+ j += 1
147
+ if p1 in panels_v2:
148
+ known_panels[0].append(j)
149
+ j = -1
150
+ for p2 in panels_v2:
151
+ j += 1
152
+ if p2 in panels_v1:
153
+ known_panels[1].append(j)
154
+
155
+ images_dir = 'urls'
156
+ if file_or_dir != 'urls':
157
+ images_dir = file_or_dir if os.path.isdir(file_or_dir) else os.path.dirname(file_or_dir)
158
+ images_dir = os.path.relpath(images_dir, 'tests/results') + '/'
159
+
160
+ diff_numbering = []
161
+ diff_panels = False
162
+ if len(known_panels[0]) != len(panels_v1) or len(known_panels[1]) != len(panels_v2):
163
+ diff_panels = True
164
+ else:
165
+ for i in range(len(panels_v1)):
166
+ if panels_v1[i] != panels_v2[i]:
167
+ diff_numbering.append(i + 1)
168
+
169
+ if diff_panels or len(diff_numbering) > 0:
170
+ files_diff[json1[p]['filename']] = {
171
+ 'jsons': [[json1[p]], [json2[p]]],
172
+ 'images_dir': images_dir,
173
+ 'known_panels': [json.dumps(known_panels[0]),
174
+ json.dumps(known_panels[1])],
175
+ 'diff_numbering_panels': diff_numbering,
176
+ }
177
+
178
+ return files_diff
179
+
180
+ @staticmethod
181
+ def draw_contours(contours, colour = 'auto', with_hull = False):
182
+ if not Debug.debug:
183
+ return
184
+
185
+ if Debug.contour_size is None:
186
+ raise Exception("Fatal error, Debug.contour_size has not been defined")
187
+
188
+ for i in range(len(contours)):
189
+ if colour == 'auto':
190
+ colour = Debug.subpanel_colours[i % len(Debug.subpanel_colours)]
191
+
192
+ cv.drawContours(Debug.img, [contours[i]], 0, colour, Debug.contour_size)
193
+
194
+ if with_hull:
195
+ hull = cv.convexHull(contours[i])
196
+ cv.drawContours(Debug.img, [hull], 0, Debug.colours['yellow'], Debug.contour_size)
197
+
198
+ @staticmethod
199
+ def draw_segments(segments, colour, size = None):
200
+ if not Debug.debug:
201
+ return
202
+
203
+ if size is None:
204
+ size = Debug.contour_size
205
+
206
+ for segment in segments:
207
+ Debug.draw_line(segment.a, segment.b, colour, size = size)
208
+
209
+ @staticmethod
210
+ def draw_line(dot1, dot2, colour, size = None):
211
+ if not Debug.debug:
212
+ return
213
+
214
+ if Debug.contour_size is None:
215
+ raise Exception("Fatal error, Debug.contour_size has not been defined")
216
+
217
+ if size is None:
218
+ size = Debug.contour_size
219
+ cv.line(Debug.img, (dot1[0], dot1[1]), (dot2[0], dot2[1]), colour, size, cv.LINE_AA)
220
+
221
+ @staticmethod
222
+ def draw_dots(dots, colour):
223
+ if not Debug.debug:
224
+ return
225
+
226
+ for dot in dots:
227
+ Debug.draw_dot(dot[0], dot[1], colour)
228
+
229
+ @staticmethod
230
+ def draw_nearby_dots(polygon, nearby_dots):
231
+ if not Debug.debug:
232
+ return
233
+
234
+ for dots in nearby_dots:
235
+ dot1 = polygon[dots[0]][0]
236
+ dot2 = polygon[dots[1]][0]
237
+ Debug.draw_dot(dot1[0], dot1[1], Debug.colours['lightpurple'])
238
+ Debug.draw_dot(dot2[0], dot2[1], Debug.colours['lightpurple'])
239
+ Debug.draw_line(dot1, dot2, Debug.colours['lightpurple'], size = 1)
240
+
241
+ @staticmethod
242
+ def draw_dot(x, y, colour):
243
+ if not Debug.debug:
244
+ return
245
+
246
+ if Debug.contour_size is None:
247
+ raise Exception("Fatal error, Debug.contour_size has not been defined")
248
+
249
+ cv.circle(Debug.img, (x, y), Debug.contour_size * 2, colour, -1)
250
+
251
+ @staticmethod
252
+ def draw_panels(panels, colour):
253
+ if not Debug.debug:
254
+ return
255
+
256
+ if Debug.contour_size is None:
257
+ raise Exception("Fatal error, Debug.contour_size has not been defined")
258
+
259
+ for p in panels:
260
+ cv.rectangle(Debug.img, (p.x, p.y), (p.r, p.b), colour, Debug.contour_size)
261
+
262
+ # + draw inner white border
263
+ for p in panels:
264
+ cv.rectangle(
265
+ Debug.img, (p.x + Debug.contour_size, p.y + Debug.contour_size),
266
+ (p.r - Debug.contour_size, p.b - Debug.contour_size), Debug.colours['white'],
267
+ int(Debug.contour_size / 2)
268
+ )
269
+
270
+ @staticmethod
271
+ def draw_polygon(polygon):
272
+ if not Debug.debug:
273
+ return
274
+
275
+ for i in range(len(polygon)):
276
+ j = (i + 1) % len(polygon)
277
+ dot1 = polygon[i][0]
278
+ dot2 = polygon[j][0]
279
+ Debug.draw_line(dot1, dot2, Debug.colours['red'], size = 2)
280
+ Debug.draw_dot(dot1[0], dot1[1], Debug.colours['gray'])
lib/html.py ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+
3
+
4
+ class HTML:
5
+
6
+ @staticmethod
7
+ def header(title = '', reldir = ''):
8
+ return f"""<!DOCTYPE html>
9
+ <html>
10
+
11
+ <head>
12
+ <title>Kumiko Reader</title>
13
+ <meta charset="utf-8">
14
+ <meta name="viewport" content="width=device-width, initial-scale=1">
15
+ <script type="text/javascript" src="{reldir}jquery-3.2.1.min.js"></script>
16
+ <script type="text/javascript" src="{reldir}reader.js"></script>
17
+ <link rel="stylesheet" media="all" href="{reldir}style.css" />
18
+ <style type="text/css">
19
+ h2, h3 {{ text-align: center; margin-top: 3em; }}
20
+ .sidebyside {{ display: flex; justify-content: space-around; }}
21
+ .sidebyside > div {{ width: 45%; }}
22
+ .version, .step-info {{ text-align: center; }}
23
+ .kumiko-reader.halfwidth {{ max-width: 45%; max-height: 90vh; }}
24
+ .kumiko-reader.fullpage {{ width: 100%; height: 100%; }}
25
+ </style>
26
+ </head>
27
+
28
+ <body>
29
+ <h1>{title}</h1>
30
+
31
+ """
32
+
33
+ @staticmethod
34
+ def nbdiffs(files_diff):
35
+ return f"<p>{len(files_diff)} differences found in files</p>"
36
+
37
+ pageId = 0
38
+
39
+ @staticmethod
40
+ def side_by_side_panels(title, step_info, jsons, v1, v2, images_dir, known_panels, diff_numbering_panels):
41
+ html = f"""
42
+ <h2>{title}</h2>
43
+ <p class="step-info">{step_info}</p>
44
+ <div class="sidebyside">
45
+ <div class="version">{v1} <span class="processing_time">− processing time {jsons[0][0]['processing_time'] if 'processing_time' in jsons[0][0] else "??"}s</span></div>
46
+ <div class="version">{v2} <span class="processing_time">- processing time {jsons[1][0]['processing_time']}s</span></div>
47
+ </div>
48
+ <div class="sidebyside">
49
+ """
50
+
51
+ oneside = """
52
+ <div id="page{id}" class="kumiko-reader halfwidth debug"></div>
53
+ <script type="text/javascript">
54
+ var reader = new Reader({{
55
+ container: $('#page{id}'),
56
+ comicsJson: {json},
57
+ images_dir: {images_dir},
58
+ known_panels: {known_panels},
59
+ diff_numbering_panels: {diff_numbering_panels},
60
+ }});
61
+ reader.start();
62
+ </script>
63
+ """
64
+ i = -1
65
+ for js in jsons:
66
+ i += 1
67
+ html += oneside.format(
68
+ id = HTML.pageId,
69
+ json = json.dumps(js),
70
+ images_dir = json.dumps(images_dir),
71
+ known_panels = known_panels[i],
72
+ diff_numbering_panels = diff_numbering_panels
73
+ )
74
+ HTML.pageId += 1
75
+
76
+ html += '</div>'
77
+ return html
78
+
79
+ @staticmethod
80
+ def imgbox(images):
81
+ html = "<h3>Debugging images</h3>\n<div class='imgbox'>\n"
82
+ for img in images:
83
+ html += f"\t<div><p>{img['label']}</p><img src='{img['filename']}' /></div>\n"
84
+
85
+ return html + "</div>\n\n"
86
+
87
+ @staticmethod
88
+ def reader(js, images_dir):
89
+ return f"""
90
+ <div id="reader" class="kumiko-reader fullpage"></div>
91
+ <script type="text/javascript">
92
+ var reader = new Reader({{
93
+ container: $('#reader'),
94
+ comicsJson: {js},
95
+ images_dir: {json.dumps(images_dir)},
96
+ controls: true
97
+ }});
98
+ reader.start();
99
+ </script>
100
+ """
101
+
102
+ footer = """
103
+
104
+ </body>
105
+ </html>
106
+ """
lib/page.py ADDED
@@ -0,0 +1,404 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import math
2
+ import os
3
+ import json
4
+ import sys
5
+ import time
6
+ import cv2 as cv
7
+ import numpy as np
8
+
9
+ from lib.panel import Panel
10
+ from lib.segment import Segment
11
+ from lib.debug import Debug
12
+
13
+
14
+ class NotAnImageException(Exception):
15
+ pass
16
+
17
+
18
+ class Page:
19
+
20
+ DEFAULT_MIN_PANEL_SIZE_RATIO = 1 / 10
21
+
22
+ def get_infos(self):
23
+ actual_gutters = self.actual_gutters()
24
+
25
+ return {
26
+ 'filename': self.url if self.url else os.path.basename(self.filename),
27
+ 'size': self.img_size,
28
+ 'numbering': self.numbering,
29
+ 'gutters': [actual_gutters['x'], actual_gutters['y']],
30
+ 'license': self.license,
31
+ 'panels': list(map(lambda p: p.to_xywh(), self.panels)),
32
+ 'processing_time': self.processing_time
33
+ }
34
+
35
+ def __init__(
36
+ self,
37
+ filename,
38
+ numbering = None,
39
+ debug = False,
40
+ url = None,
41
+ min_panel_size_ratio = None,
42
+ panel_expansion = True
43
+ ):
44
+ self.filename = filename
45
+ self.panels = []
46
+ self.segments = []
47
+
48
+ self.processing_time = None
49
+ t1 = time.time_ns()
50
+
51
+ self.img = cv.imread(filename)
52
+ if not isinstance(self.img, np.ndarray) or self.img.size == 0:
53
+ raise NotAnImageException(f"File {filename} is not an image")
54
+
55
+ self.numbering = numbering or "ltr"
56
+ if not (numbering in ['ltr', 'rtl']):
57
+ raise Exception('Fatal error, unknown numbering: ' + str(numbering))
58
+
59
+ self.small_panel_ratio = min_panel_size_ratio or Page.DEFAULT_MIN_PANEL_SIZE_RATIO
60
+ self.panel_expansion = panel_expansion
61
+ self.url = url
62
+
63
+ self.img_size = list(self.img.shape[:2])
64
+ self.img_size.reverse() # get a [width,height] list
65
+
66
+ Debug.contour_size = 3
67
+
68
+ # get license for this file
69
+ self.license = None
70
+ if os.path.isfile(filename + '.license'):
71
+ with open(filename + '.license', encoding = "utf8") as fh:
72
+ try:
73
+ self.license = json.load(fh)
74
+ except json.decoder.JSONDecodeError:
75
+ print(f"License file {filename+'.license'} is not a valid JSON file", file = sys.stderr)
76
+ sys.exit(1)
77
+
78
+ Debug.set_base_img(self.img)
79
+
80
+ Debug.add_step('Initial state', self.get_infos())
81
+ Debug.add_image('Input image')
82
+
83
+ self.gray = cv.cvtColor(self.img, cv.COLOR_BGR2GRAY)
84
+ Debug.add_image('Shades of gray', img = self.gray)
85
+ Debug.show_time("Shades of gray")
86
+
87
+ # https://docs.opencv.org/3.4/d2/d2c/tutorial_sobel_derivatives.html
88
+ ddepth = cv.CV_16S
89
+ grad_x = cv.Sobel(self.gray, ddepth, 1, 0, ksize = 3, scale = 1, delta = 0, borderType = cv.BORDER_DEFAULT)
90
+ # Gradient-Y
91
+ # grad_y = cv.Scharr(self.gray,ddepth,0,1)
92
+ grad_y = cv.Sobel(self.gray, ddepth, 0, 1, ksize = 3, scale = 1, delta = 0, borderType = cv.BORDER_DEFAULT)
93
+
94
+ abs_grad_x = cv.convertScaleAbs(grad_x)
95
+ abs_grad_y = cv.convertScaleAbs(grad_y)
96
+
97
+ self.sobel = cv.addWeighted(abs_grad_x, 0.5, abs_grad_y, 0.5, 0)
98
+ Debug.add_image('Sobel filter applied', img = self.sobel)
99
+ Debug.show_time("Sobel filter")
100
+
101
+ self.get_contours()
102
+ self.get_segments()
103
+ self.get_initial_panels()
104
+ self.group_small_panels()
105
+ self.split_panels()
106
+ self.exclude_small_panels()
107
+ self.merge_panels()
108
+ self.deoverlap_panels()
109
+ self.exclude_small_panels()
110
+
111
+ if self.panel_expansion:
112
+ self.panels.sort() # TODO: move this below before panels sort-fix, when panels expansion is smarter
113
+ self.expand_panels()
114
+
115
+ if len(self.panels) == 0:
116
+ self.panels.append(Panel(page = self, xywh = [0, 0, self.img_size[0], self.img_size[1]]))
117
+
118
+ self.group_big_panels()
119
+
120
+ self.fix_panels_numbering()
121
+
122
+ self.processing_time = int((time.time_ns() - t1) / 10**7) / 100
123
+
124
+ def get_contours(self):
125
+ # Black background: values above 100 will be black, the rest white
126
+ _, thresh = cv.threshold(self.sobel, 100, 255, cv.THRESH_BINARY)
127
+ Debug.show_time("Image threshhold")
128
+
129
+ self.contours, _ = cv.findContours(thresh, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)[-2:]
130
+
131
+ Debug.add_image("Thresholded image", img = thresh)
132
+ Debug.show_time("Get contours")
133
+
134
+ def get_segments(self):
135
+ self.segments = None
136
+
137
+ lsd = cv.createLineSegmentDetector(0)
138
+ dlines = lsd.detect(self.gray)
139
+
140
+ Debug.show_time("Detected segments")
141
+
142
+ min_dist = min(self.img_size) * self.small_panel_ratio
143
+
144
+ while self.segments is None or len(self.segments) > 500:
145
+ self.segments = []
146
+
147
+ if dlines is None or dlines[0] is None:
148
+ break
149
+
150
+ for dline in dlines[0]:
151
+ x0 = int(round(dline[0][0]))
152
+ y0 = int(round(dline[0][1]))
153
+ x1 = int(round(dline[0][2]))
154
+ y1 = int(round(dline[0][3]))
155
+
156
+ a = x0 - x1
157
+ b = y0 - y1
158
+ dist = math.sqrt(a**2 + b**2)
159
+ if dist >= min_dist:
160
+ self.segments.append(Segment([x0, y0], [x1, y1]))
161
+
162
+ min_dist *= 1.1
163
+
164
+ self.segments = Segment.union_all(self.segments)
165
+
166
+ Debug.draw_segments(self.segments, Debug.colours['green'])
167
+ Debug.add_image("Segment Detector")
168
+ Debug.show_time("Compiled segments")
169
+
170
+ # Get (square) panels out of initial contours
171
+ def get_initial_panels(self):
172
+ self.panels = []
173
+ for contour in self.contours:
174
+ arclength = cv.arcLength(contour, True)
175
+ epsilon = 0.001 * arclength
176
+ approx = cv.approxPolyDP(contour, epsilon, True)
177
+
178
+ panel = Panel(page = self, polygon = approx)
179
+ if panel.is_very_small():
180
+ continue
181
+
182
+ Debug.draw_contours([approx], Debug.colours['red'])
183
+
184
+ self.panels.append(panel)
185
+
186
+ Debug.add_image('Initial contours')
187
+ Debug.add_step('Panels from initial contours', self.get_infos())
188
+
189
+ # Group small panels that are close together, into bigger ones
190
+ def group_small_panels(self):
191
+ small_panels = list(filter(lambda p: p.is_small(), self.panels))
192
+ groups = {}
193
+ group_id = 0
194
+
195
+ for i, p1 in enumerate(small_panels):
196
+ for p2 in small_panels[i + 1:]:
197
+ if p1 == p2:
198
+ continue
199
+
200
+ if not p1.is_close(p2):
201
+ continue
202
+
203
+ if p1 not in groups and p2 not in groups:
204
+ group_id += 1
205
+ groups[p1] = group_id
206
+ groups[p2] = group_id
207
+ elif p1 in groups and p2 not in groups:
208
+ groups[p2] = groups[p1]
209
+ elif p2 in groups and p1 not in groups:
210
+ groups[p1] = groups[p2]
211
+ elif groups[p1] != groups[p2]:
212
+ # group group1 and group2 together
213
+ for p, id in groups.items():
214
+ if id == groups[p2]:
215
+ groups[p] = groups[p1]
216
+
217
+ grouped = {}
218
+ for k, v in groups.items():
219
+ grouped[v] = grouped.get(v, []) + [k]
220
+
221
+ for small_panels in grouped.values():
222
+ big_hull = cv.convexHull(np.concatenate(list(map(lambda p: p.polygon, small_panels))))
223
+ big_panel = Panel(page = self, polygon = big_hull, splittable = False)
224
+
225
+ self.panels.append(big_panel)
226
+ for p in small_panels:
227
+ self.panels.remove(p)
228
+
229
+ Debug.draw_contours(list(map(lambda p: p.polygon, small_panels)), Debug.colours['lightblue'])
230
+ Debug.draw_contours([big_panel.polygon], Debug.colours['red'])
231
+
232
+ if group_id > 0:
233
+ Debug.add_image('Group small panels')
234
+ Debug.add_step('Group small panels', self.get_infos())
235
+
236
+ # See if panels can be cut into several (two non-consecutive points are close)
237
+ def split_panels(self):
238
+ did_split = True
239
+ while did_split:
240
+ did_split = False
241
+ for p in sorted(self.panels, key = lambda p: p.area(), reverse = True):
242
+ split = p.split()
243
+ if split is not None:
244
+ did_split = True
245
+ self.panels.remove(p)
246
+ self.panels += split.subpanels
247
+
248
+ Debug.draw_contours(list(map(lambda n: n.polygon, split.subpanels)), Debug.colours['blue'])
249
+ Debug.draw_line(split.segment.a, split.segment.b, Debug.colours['red'])
250
+ break
251
+
252
+ if did_split:
253
+ Debug.add_image(
254
+ 'Split contours (blue contours, red split-segment, gray polygon dots, purple nearby dots)'
255
+ )
256
+
257
+ Debug.add_step(f"Panels from split contours ({len(self.segments)} segments)", self.get_infos())
258
+
259
+ def exclude_small_panels(self):
260
+ self.panels = list(filter(lambda p: not p.is_small(), self.panels))
261
+
262
+ Debug.add_step('Exclude small panels', self.get_infos())
263
+
264
+ # Splitting polygons may result in panels slightly overlapping, de-overlap them
265
+ def deoverlap_panels(self):
266
+ for p1 in self.panels:
267
+ for p2 in self.panels:
268
+ if p1 == p2:
269
+ continue
270
+
271
+ opanel = p1.overlap_panel(p2)
272
+ if not opanel:
273
+ continue
274
+
275
+ if opanel.w() < opanel.h() and p1.r == opanel.r:
276
+ p1.r = opanel.x
277
+ p2.x = opanel.r
278
+ continue
279
+
280
+ if opanel.w() > opanel.h() and p1.b == opanel.b:
281
+ p1.b = opanel.y
282
+ p2.y = opanel.b
283
+ continue
284
+
285
+ Debug.add_step('Deoverlap panels', self.get_infos())
286
+
287
+ # Merge panels that shouldn't have been split (speech bubble diving into a panel)
288
+ def merge_panels(self):
289
+ panels_to_remove = []
290
+ for i, p1 in enumerate(self.panels):
291
+ for j, p2 in enumerate(self.panels[i + 1:]):
292
+ if p1.contains(p2):
293
+ panels_to_remove.append(p2)
294
+ p1 = p1.merge(p2)
295
+ elif p2.contains(p1):
296
+ panels_to_remove.append(p1)
297
+ p2 = p2.merge(p1)
298
+
299
+ for p in set(panels_to_remove):
300
+ self.panels.remove(p)
301
+
302
+ Debug.add_step('Merge panels', self.get_infos())
303
+
304
+ # Find out actual gutters between panels
305
+ def actual_gutters(self, func = min):
306
+ gutters_x = []
307
+ gutters_y = []
308
+ for p in self.panels:
309
+ left_panel = p.find_left_panel()
310
+ if left_panel:
311
+ gutters_x.append(p.x - left_panel.r)
312
+
313
+ top_panel = p.find_top_panel()
314
+ if top_panel:
315
+ gutters_y.append(p.y - top_panel.b)
316
+
317
+ if not gutters_x:
318
+ gutters_x = [1]
319
+ if not gutters_y:
320
+ gutters_y = [1]
321
+
322
+ return {'x': func(gutters_x), 'y': func(gutters_y), 'r': -func(gutters_x), 'b': -func(gutters_y)}
323
+
324
+ def max_gutter(self):
325
+ return max(self.actual_gutters().values())
326
+
327
+ # Expand panels to their neighbour's edge, or page boundaries
328
+ def expand_panels(self):
329
+ gutters = self.actual_gutters()
330
+ for p in self.panels:
331
+ for d in ['x', 'y', 'r', 'b']: # expand in all four directions
332
+ newcoord = -1
333
+ neighbour = p.find_neighbour_panel(d)
334
+ if neighbour:
335
+ # expand to that neighbour's edge (minus gutter)
336
+ newcoord = getattr(neighbour, {'x': 'r', 'r': 'x', 'y': 'b', 'b': 'y'}[d]) + gutters[d]
337
+ else:
338
+ # expand to the furthest known edge (frame around all panels)
339
+ min_panel = min(self.panels, key = lambda p: getattr(p, d)) if d in [
340
+ 'x', 'y'
341
+ ] else max(self.panels, key = lambda p: getattr(p, d))
342
+ newcoord = getattr(min_panel, d)
343
+
344
+ if newcoord != -1:
345
+ if d in ['r', 'b'] and newcoord > getattr(p, d) or d in ['x', 'y'] and newcoord < getattr(p, d):
346
+ setattr(p, d, newcoord)
347
+
348
+ Debug.add_step('Expand panels', self.get_infos())
349
+
350
+ # Fix panels simple sorting (issue #12)
351
+ def fix_panels_numbering(self):
352
+ changes = 1
353
+ while changes:
354
+ changes = 0
355
+ for i, p in enumerate(self.panels):
356
+ neighbours_before = [p.find_top_panel()]
357
+ neighbours_before += p.find_all_right_panels() if self.numbering == "rtl" else p.find_all_left_panels()
358
+
359
+ for neighbour in neighbours_before:
360
+ if neighbour is None:
361
+ continue
362
+ neighbour_pos = self.panels.index(neighbour)
363
+ if i < neighbour_pos:
364
+ changes += 1
365
+ self.panels.insert(neighbour_pos, self.panels.pop(i))
366
+ break
367
+ if changes > 0:
368
+ break # start a new whole loop with reordered panels
369
+
370
+ Debug.add_step('Numbering fixed', self.get_infos())
371
+
372
+ # group big panels together
373
+ def group_big_panels(self):
374
+ grouped = True
375
+ while grouped:
376
+ grouped = False
377
+ for i, p1 in enumerate(self.panels):
378
+ for p2 in self.panels[i + 1:]:
379
+ p3 = p1.group_with(p2)
380
+
381
+ other_panels = [p for p in self.panels if p not in [p1, p2]]
382
+ if p3.bumps_into(other_panels):
383
+ continue
384
+
385
+ # are there big segments in this panel?
386
+ segments = []
387
+ for s in self.segments:
388
+ if p3.contains_segment(s) and s.dist() > p3.diagonal().dist() / 5:
389
+ if s not in segments:
390
+ segments.append(s)
391
+
392
+ if len(segments) > 0: # maybe allow a small number of big segments here?
393
+ continue
394
+
395
+ self.panels.append(p3)
396
+ self.panels.remove(p1)
397
+ self.panels.remove(p2)
398
+ grouped = True
399
+ break
400
+
401
+ if grouped:
402
+ break
403
+
404
+ Debug.add_step('Group big panels', self.get_infos())
lib/panel.py ADDED
@@ -0,0 +1,479 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import math
2
+ import cv2 as cv
3
+ import numpy as np
4
+
5
+ from lib.segment import Segment
6
+ from lib.debug import Debug
7
+
8
+
9
+ class Panel:
10
+
11
+ @staticmethod
12
+ def from_xyrb(page, x, y, r, b):
13
+ return Panel(page, xywh = [x, y, r - x, b - y])
14
+
15
+ def __init__(self, page, xywh = None, polygon = None, splittable = True):
16
+ self.page = page
17
+
18
+ if xywh is None and polygon is None:
19
+ raise Exception('Fatal error: no parameter to define Panel boundaries')
20
+
21
+ if xywh is None:
22
+ xywh = cv.boundingRect(polygon)
23
+
24
+ self.x = xywh[0] # panel's left edge
25
+ self.y = xywh[1] # panel's top edge
26
+ self.r = self.x + xywh[2] # panel's right edge
27
+ self.b = self.y + xywh[3] # panel's bottom edge
28
+
29
+ self.polygon = polygon
30
+ self.splittable = splittable
31
+ self.segments = None
32
+ self.coverage = None
33
+
34
+ def w(self):
35
+ return self.r - self.x
36
+
37
+ def h(self):
38
+ return self.b - self.y
39
+
40
+ def diagonal(self):
41
+ return Segment((self.x, self.y), (self.r, self.b))
42
+
43
+ def wt(self):
44
+ return self.w() / 10
45
+ # wt = width threshold (under which two edge coordinates are considered equal)
46
+
47
+ def ht(self):
48
+ return self.h() / 10
49
+ # ht = height threshold
50
+
51
+ def to_xywh(self):
52
+ return [self.x, self.y, self.w(), self.h()]
53
+
54
+ def __eq__(self, other):
55
+ return all(
56
+ [
57
+ abs(self.x - other.x) < self.wt(),
58
+ abs(self.y - other.y) < self.ht(),
59
+ abs(self.r - other.r) < self.wt(),
60
+ abs(self.b - other.b) < self.ht(),
61
+ ]
62
+ )
63
+
64
+ def __lt__(self, other):
65
+ # panel is above other
66
+ if other.y >= self.b - self.ht() and other.y >= self.y - self.ht():
67
+ return True
68
+
69
+ # panel is below other
70
+ if self.y >= other.b - self.ht() and self.y >= other.y - self.ht():
71
+ return False
72
+
73
+ # panel is left from other
74
+ if other.x >= self.r - self.wt() and other.x >= self.x - self.wt():
75
+ return True if self.page.numbering == 'ltr' else False
76
+
77
+ # panel is right from other
78
+ if self.x >= other.r - self.wt() and self.x >= other.x - self.wt():
79
+ return False if self.page.numbering == 'ltr' else True
80
+
81
+ return True # should not happen, TODO: raise an exception?
82
+
83
+ def __le__(self, other):
84
+ return self.__lt__(other)
85
+
86
+ def __gt__(self, other):
87
+ return not self.__lt__(other)
88
+
89
+ def __ge__(self, other):
90
+ return self.__gt__(other)
91
+
92
+ def area(self):
93
+ return self.w() * self.h()
94
+
95
+ def __str__(self):
96
+ return f"{self.x}x{self.y}-{self.r}x{self.b}"
97
+
98
+ def __hash__(self):
99
+ return hash(self.__str__())
100
+
101
+ def is_small(self, extra_ratio = 1):
102
+ return any(
103
+ [
104
+ self.w() < self.page.img_size[0] * self.page.small_panel_ratio * extra_ratio,
105
+ self.h() < self.page.img_size[1] * self.page.small_panel_ratio * extra_ratio,
106
+ ]
107
+ )
108
+
109
+ def is_very_small(self):
110
+ return self.is_small(1 / 10)
111
+
112
+ def overlap_panel(self, other):
113
+ if self.x > other.r or other.x > self.r: # panels are left and right from one another
114
+ return None
115
+ if self.y > other.b or other.y > self.b: # panels are above and below one another
116
+ return None
117
+
118
+ # if we're here, panels overlap at least a bit
119
+ x = max(self.x, other.x)
120
+ y = max(self.y, other.y)
121
+ r = min(self.r, other.r)
122
+ b = min(self.b, other.b)
123
+
124
+ return Panel(self.page, [x, y, r - x, b - y])
125
+
126
+ def overlap_area(self, other):
127
+ opanel = self.overlap_panel(other)
128
+ if opanel is None:
129
+ return 0
130
+
131
+ return opanel.area()
132
+
133
+ def overlaps(self, other):
134
+ opanel = self.overlap_panel(other)
135
+ if opanel is None:
136
+ return False
137
+
138
+ area_ratio = 0.1
139
+ smallest_panel_area = min(self.area(), other.area())
140
+
141
+ if smallest_panel_area == 0: # probably a horizontal or vertical segment
142
+ return True
143
+
144
+ return opanel.area() / smallest_panel_area > area_ratio
145
+
146
+ def contains(self, other):
147
+ o_panel = self.overlap_panel(other)
148
+ if not o_panel:
149
+ return False
150
+
151
+ # self contains other if their overlapping area is more than 50% of other's area
152
+ return o_panel.area() / other.area() > 0.50
153
+
154
+ def same_row(self, other):
155
+ above, below = sorted([self, other], key = lambda p: p.y)
156
+
157
+ if below.y > above.b: # stricly above
158
+ return False
159
+
160
+ if below.b < above.b: # contained
161
+ return True
162
+
163
+ # intersect
164
+ intersection_y = min(above.b, below.b) - below.y
165
+ min_h = min(above.h(), below.h())
166
+ return min_h == 0 or intersection_y / min_h >= 1 / 3
167
+
168
+ def same_col(self, other):
169
+ left, right = sorted([self, other], key = lambda p: p.x)
170
+
171
+ if right.x > left.r: # stricly left
172
+ return False
173
+
174
+ if right.r < left.r: # contained
175
+ return True
176
+
177
+ # intersect
178
+ intersection_x = min(left.r, right.r) - right.x
179
+ min_w = min(left.w(), right.w())
180
+ return min_w == 0 or intersection_x / min_w >= 1 / 3
181
+
182
+ def find_top_panel(self):
183
+ all_top = list(filter(lambda p: p.b <= self.y and p.same_col(self), self.page.panels))
184
+ return max(all_top, key = lambda p: p.b) if all_top else None
185
+
186
+ def find_bottom_panel(self):
187
+ all_bottom = list(filter(lambda p: p.y >= self.b and p.same_col(self), self.page.panels))
188
+ return min(all_bottom, key = lambda p: p.y) if all_bottom else None
189
+
190
+ def find_all_left_panels(self):
191
+ return list(filter(lambda p: p.r <= self.x and p.same_row(self), self.page.panels))
192
+
193
+ def find_left_panel(self):
194
+ all_left = self.find_all_left_panels()
195
+ return max(all_left, key = lambda p: p.r) if all_left else None
196
+
197
+ def find_all_right_panels(self):
198
+ return list(filter(lambda p: p.x >= self.r and p.same_row(self), self.page.panels))
199
+
200
+ def find_right_panel(self):
201
+ all_right = self.find_all_right_panels()
202
+ return min(all_right, key = lambda p: p.x) if all_right else None
203
+
204
+ def find_neighbour_panel(self, d):
205
+ return {
206
+ 'x': self.find_left_panel,
207
+ 'y': self.find_top_panel,
208
+ 'r': self.find_right_panel,
209
+ 'b': self.find_bottom_panel,
210
+ }[d]()
211
+
212
+ def group_with(self, other):
213
+ min_x = min(self.x, other.x)
214
+ min_y = min(self.y, other.y)
215
+ max_r = max(self.r, other.r)
216
+ max_b = max(self.b, other.b)
217
+ return Panel(self.page, [min_x, min_y, max_r - min_x, max_b - min_y])
218
+
219
+ def merge(self, other):
220
+ possible_panels = [self]
221
+
222
+ # expand self in all four directions where other is
223
+ if other.x < self.x:
224
+ possible_panels.append(Panel.from_xyrb(self.page, other.x, self.y, self.r, self.b))
225
+
226
+ if other.r > self.r:
227
+ for pp in possible_panels.copy():
228
+ possible_panels.append(Panel.from_xyrb(self.page, pp.x, pp.y, other.r, pp.b))
229
+
230
+ if other.y < self.y:
231
+ for pp in possible_panels.copy():
232
+ possible_panels.append(Panel.from_xyrb(self.page, pp.x, other.y, pp.r, pp.b))
233
+
234
+ if other.b > self.b:
235
+ for pp in possible_panels.copy():
236
+ possible_panels.append(Panel.from_xyrb(self.page, pp.x, pp.y, pp.r, other.b))
237
+
238
+ # don't take a merged panel that bumps into other panels on page
239
+ other_panels = [p for p in self.page.panels if p not in [self, other]]
240
+ possible_panels = list(filter(lambda p: not p.bumps_into(other_panels), possible_panels))
241
+
242
+ # take the largest merged panel
243
+ return max(possible_panels, key = lambda p: p.area()) if len(possible_panels) > 0 else self
244
+
245
+ def is_close(self, other):
246
+ c1x = self.x + self.w() / 2
247
+ c1y = self.y + self.h() / 2
248
+ c2x = other.x + other.w() / 2
249
+ c2y = other.y + other.h() / 2
250
+
251
+ return all(
252
+ [
253
+ abs(c1x - c2x) <= (self.w() + other.w()) * 0.75,
254
+ abs(c1y - c2y) <= (self.h() + other.h()) * 0.75,
255
+ ]
256
+ )
257
+
258
+ def bumps_into(self, other_panels):
259
+ for other in other_panels:
260
+ if other == self:
261
+ continue
262
+ if self.overlaps(other):
263
+ return True
264
+
265
+ return False
266
+
267
+ def contains_segment(self, segment):
268
+ other = Panel.from_xyrb(None, *segment.to_xyrb())
269
+ return self.overlaps(other)
270
+
271
+ def get_segments(self):
272
+ if self.segments is not None:
273
+ return self.segments
274
+
275
+ self.segments = list(filter(lambda s: self.contains_segment(s), self.page.segments))
276
+
277
+ return self.segments
278
+
279
+ def split(self):
280
+ if self.splittable is False:
281
+ return None
282
+
283
+ split = self._cached_split()
284
+
285
+ if split is None:
286
+ self.splittable = False
287
+
288
+ return split
289
+
290
+ def _cached_split(self):
291
+ if self.polygon is None:
292
+ return None
293
+
294
+ if self.is_small(extra_ratio = 2): # panel should be splittable in two non-small subpanels
295
+ return None
296
+
297
+ min_hops = 3
298
+ max_dist_x = int(self.w() / 3)
299
+ max_dist_y = int(self.h() / 3)
300
+ max_diagonal = math.sqrt(max_dist_x**2 + max_dist_y**2)
301
+ dots_along_lines_dist = max_diagonal / 5
302
+ min_dist_between_dots_x = max_dist_x / 10
303
+ min_dist_between_dots_y = max_dist_y / 10
304
+
305
+ # Compose modified polygon to optimise splits
306
+ original_polygon = np.copy(self.polygon)
307
+ polygon = np.ndarray(shape = (0, 1, 2), dtype = int, order = 'F')
308
+ intermediary_dots = []
309
+ extra_dots = []
310
+
311
+ for i in range(len(original_polygon)):
312
+ j = (i + 1) % len(original_polygon)
313
+ dot1 = tuple(original_polygon[i][0])
314
+ dot2 = tuple(original_polygon[j][0])
315
+ seg = Segment(dot1, dot2)
316
+
317
+ # merge nearby dots together
318
+ if seg.dist_x() < min_dist_between_dots_x and seg.dist_y() < min_dist_between_dots_y:
319
+ original_polygon[j][0] = seg.center()
320
+ continue
321
+
322
+ polygon = np.append(polygon, [[dot1]], axis = 0)
323
+
324
+ # Add dots on *long* edges, by projecting other polygon dots on this segment
325
+ add_dots = []
326
+
327
+ # should be splittable in [dot1, dot1b(?), projected_dot3, dot2b(?), dot2]
328
+ if seg.dist() < dots_along_lines_dist * 2:
329
+ continue
330
+
331
+ for k, dot3 in enumerate(original_polygon):
332
+ if abs(k - i) < min_hops:
333
+ continue
334
+
335
+ projected_dot3 = seg.projected_point(dot3)
336
+
337
+ # Segment should be able to contain projected_dot3
338
+ if not seg.may_contain(projected_dot3):
339
+ continue
340
+
341
+ # dot3 should be close to current segment − distance(dot3, projected_dot3) should be short
342
+ project = Segment(dot3[0], projected_dot3)
343
+ if project.dist_x() > max_dist_x or project.dist_y() > max_dist_y:
344
+ continue
345
+
346
+ # append dot3 as intermediary dot on segment(dot1, dot2)
347
+ add_dots.append(projected_dot3)
348
+ intermediary_dots.append(projected_dot3)
349
+
350
+ # Add also a dot near each end of the segment (provoke segment matching)
351
+ alpha_x = math.acos(seg.dist_x(keep_sign = True) / seg.dist())
352
+ alpha_y = math.asin(seg.dist_y(keep_sign = True) / seg.dist())
353
+ dist_x = int(math.cos(alpha_x) * dots_along_lines_dist)
354
+ dist_y = int(math.sin(alpha_y) * dots_along_lines_dist)
355
+
356
+ dot1b = (dot1[0] + dist_x, dot1[1] + dist_y)
357
+ # if len(intermediary_dots) == 0 or Segment(dot1b, intermediary_dots[0]).dist() > dots_along_lines_dist:
358
+ add_dots.append(dot1b)
359
+ extra_dots.append(dot1b)
360
+
361
+ dot2b = (dot2[0] - dist_x, dot2[1] - dist_y)
362
+ # if len(intermediary_dots) == 0 or Segment(dot2b, intermediary_dots[-1]).dist() > dots_along_lines_dist:
363
+ add_dots.append(dot2b)
364
+ extra_dots.append(dot2b)
365
+
366
+ for dot in sorted(add_dots, key = lambda dot: Segment(dot1, dot).dist()):
367
+ polygon = np.append(polygon, [[dot]], axis = 0)
368
+
369
+ # Re-merge nearby dots together
370
+ original_polygon = np.copy(polygon)
371
+ polygon = np.ndarray(shape = (0, 1, 2), dtype = int, order = 'F')
372
+
373
+ for i in range(len(original_polygon)):
374
+ j = (i + 1) % len(original_polygon)
375
+ dot1 = tuple(original_polygon[i][0])
376
+ dot2 = tuple(original_polygon[j][0])
377
+ seg = Segment(dot1, dot2)
378
+
379
+ # merge nearby dots together
380
+ if seg.dist_x() < min_dist_between_dots_x and seg.dist_y() < min_dist_between_dots_y:
381
+ intermediary_dots = [dot for dot in intermediary_dots if dot not in [dot1, dot2]]
382
+ extra_dots = [dot for dot in extra_dots if dot not in [dot1, dot2]]
383
+ original_polygon[j][0] = seg.center()
384
+ continue
385
+
386
+ polygon = np.append(polygon, [[dot1]], axis = 0)
387
+
388
+ Debug.draw_polygon(polygon)
389
+ Debug.draw_dots(intermediary_dots, Debug.colours['red'])
390
+ Debug.draw_dots(extra_dots, Debug.colours['yellow'])
391
+ Debug.add_image(f"Composed polygon {self} ({len(polygon)} dots, {len(intermediary_dots)} intermediary)")
392
+
393
+ # Find dots nearby one another
394
+ nearby_dots = []
395
+
396
+ for i in range(len(polygon) - min_hops):
397
+ for j in range(i + min_hops, len(polygon)):
398
+ dot1 = polygon[i][0]
399
+ dot2 = polygon[j][0]
400
+ seg = Segment(dot1, dot2)
401
+
402
+ if seg.dist_x() <= max_dist_x and seg.dist_y() <= max_dist_y:
403
+ nearby_dots.append([i, j])
404
+
405
+ if len(nearby_dots) == 0:
406
+ return None
407
+
408
+ Debug.draw_nearby_dots(polygon, nearby_dots)
409
+ Debug.add_image(f"Nearby dots ({len(nearby_dots)})")
410
+
411
+ splits = []
412
+ for dots in nearby_dots:
413
+ poly1len = len(polygon) - dots[1] + dots[0]
414
+ poly2len = dots[1] - dots[0]
415
+
416
+ # A panel should have at least three edges
417
+ if min(poly1len, poly2len) <= 2:
418
+ continue
419
+
420
+ # Construct two subpolygons by distributing the dots around our nearby dots
421
+ poly1 = np.zeros(shape = (poly1len, 1, 2), dtype = int)
422
+ poly2 = np.zeros(shape = (poly2len, 1, 2), dtype = int)
423
+
424
+ x = y = 0
425
+ for i in range(len(polygon)):
426
+ if i <= dots[0] or i > dots[1]:
427
+ poly1[x][0] = polygon[i]
428
+ x += 1
429
+ else:
430
+ poly2[y][0] = polygon[i]
431
+ y += 1
432
+
433
+ panel1 = Panel(self.page, polygon = poly1)
434
+ panel2 = Panel(self.page, polygon = poly2)
435
+
436
+ if panel1.is_small() or panel2.is_small():
437
+ continue
438
+
439
+ if panel1 == self or panel2 == self:
440
+ continue
441
+
442
+ if panel1.overlaps(panel2):
443
+ continue
444
+
445
+ split_segment = Segment.along_polygon(polygon, dots[0], dots[1])
446
+ split = Split(self, panel1, panel2, split_segment)
447
+ if split not in splits:
448
+ splits.append(split)
449
+
450
+ Debug.draw_segments([split.segment for split in splits], Debug.colours['red'], size = 2)
451
+ Debug.add_image(f"Splits ({len(splits)})")
452
+
453
+ splits = list(filter(lambda split: split.segments_coverage() > 50 / 100, splits))
454
+
455
+ if len(splits) == 0:
456
+ return None
457
+
458
+ # return the split that best matches segments (~panel edges)
459
+ best_split = max(splits, key = lambda split: split.covered_dist)
460
+
461
+ return best_split
462
+
463
+
464
+ class Split:
465
+
466
+ def __init__(self, panel, subpanel1, subpanel2, split_segment):
467
+ self.panel = panel
468
+ self.subpanels = [subpanel1, subpanel2]
469
+ self.segment = split_segment
470
+
471
+ self.matching_segments = self.segment.intersect_all(self.panel.get_segments())
472
+ self.covered_dist = sum(map(lambda s: s.dist(), self.matching_segments))
473
+
474
+ def __eq__(self, other):
475
+ return self.segment == other.segment
476
+
477
+ def segments_coverage(self):
478
+ segment_dist = self.segment.dist()
479
+ return self.covered_dist / segment_dist if segment_dist else 0
lib/segment.py ADDED
@@ -0,0 +1,197 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import math
2
+ import numpy as np
3
+
4
+
5
+ class Segment:
6
+
7
+ def __init__(self, a, b):
8
+ self.a = (int(a[0]), int(a[1]))
9
+ self.b = (int(b[0]), int(b[1]))
10
+
11
+ for dot in [self.a, self.b]:
12
+ if len(dot) != 2:
13
+ raise Exception(f"Creating a segment with more or less than two dots: Segment({a}, {b})")
14
+ if type(dot[0]) != int or type(dot[1]) != int:
15
+ raise Exception(f"Creating a segment with non-dots: Segment({a}, {b})")
16
+
17
+ def __str__(self):
18
+ return f"({self.a}, {self.b})"
19
+
20
+ def __eq__(self, other):
21
+ return any([
22
+ self.a == other.a and self.b == other.b,
23
+ self.a == other.b and self.b == other.a,
24
+ ])
25
+
26
+ def dist(self):
27
+ return math.sqrt(self.dist_x()**2 + self.dist_y()**2)
28
+
29
+ def dist_x(self, keep_sign = False):
30
+ dist = self.b[0] - self.a[0]
31
+ return dist if keep_sign else abs(dist)
32
+
33
+ def dist_y(self, keep_sign = False):
34
+ dist = self.b[1] - self.a[1]
35
+ return dist if keep_sign else abs(dist)
36
+
37
+ def left(self):
38
+ return min(self.a[0], self.b[0])
39
+
40
+ def top(self):
41
+ return min(self.a[1], self.b[1])
42
+
43
+ def right(self):
44
+ return max(self.a[0], self.b[0])
45
+
46
+ def bottom(self):
47
+ return max(self.a[1], self.b[1])
48
+
49
+ def to_xyrb(self):
50
+ return [self.left(), self.top(), self.right(), self.bottom()]
51
+
52
+ def center(self):
53
+ return (
54
+ int(self.left() + self.dist_x() / 2),
55
+ int(self.top() + self.dist_y() / 2),
56
+ )
57
+
58
+ def may_contain(self, dot):
59
+ return all([
60
+ dot[0] >= self.left(),
61
+ dot[0] <= self.right(),
62
+ dot[1] >= self.top(),
63
+ dot[1] <= self.bottom(),
64
+ ])
65
+
66
+ def intersect(self, other):
67
+ gutter = max(self.dist(), other.dist()) * 5 / 100
68
+
69
+ # angle too big ?
70
+ if not self.angle_ok_with(other):
71
+ return None
72
+
73
+ # from here, segments are almost parallel
74
+
75
+ # segments are apart ?
76
+ if any(
77
+ [
78
+ self.right() < other.left() - gutter, # self left from other
79
+ self.left() > other.right() + gutter, # self right from other
80
+ self.bottom() < other.top() - gutter, # self above other
81
+ self.top() > other.bottom() + gutter, # self below other
82
+ ]
83
+ ):
84
+ return None
85
+
86
+ projected_c = self.projected_point(other.a)
87
+ dist_c_to_ab = Segment(other.a, projected_c).dist()
88
+
89
+ projected_d = self.projected_point(other.b)
90
+ dist_d_to_ab = Segment(other.b, projected_d).dist()
91
+
92
+ # segments are a bit too far from each other
93
+ if (dist_c_to_ab + dist_d_to_ab) / 2 > gutter:
94
+ return None
95
+
96
+ # segments overlap, or one contains the other
97
+ # A----B
98
+ # C----D
99
+ # or
100
+ # A------------B
101
+ # C----D
102
+ sorted_dots = sorted([self.a, self.b, other.a, other.b], key = sum)
103
+ middle_dots = sorted_dots[1:3]
104
+ b, c = middle_dots
105
+
106
+ return Segment(b, c)
107
+
108
+ def union(self, other):
109
+ intersect = self.intersect(other)
110
+ if intersect is None:
111
+ return None
112
+
113
+ dots = [tuple(self.a), tuple(self.b), tuple(other.a), tuple(other.b)]
114
+ dots.remove(tuple(intersect.a))
115
+ dots.remove(tuple(intersect.b))
116
+ return Segment(dots[0], dots[1])
117
+
118
+ def angle_with(self, other):
119
+ return math.degrees(abs(self.angle() - other.angle()))
120
+
121
+ def angle_ok_with(self, other):
122
+ angle = self.angle_with(other)
123
+ return angle < 10 or abs(angle - 180) < 10
124
+
125
+ def angle(self):
126
+ return math.atan(self.dist_y() / self.dist_x()) if self.dist_x() != 0 else math.pi / 2
127
+
128
+ def intersect_all(self, segments):
129
+ segments_match = []
130
+ for segment in segments:
131
+ s3 = self.intersect(segment)
132
+ if s3 is not None:
133
+ segments_match.append(s3)
134
+
135
+ return Segment.union_all(segments_match)
136
+
137
+ @staticmethod
138
+ def along_polygon(polygon, i, j):
139
+ dot1 = polygon[i][0]
140
+ dot2 = polygon[j][0]
141
+ split_segment = Segment(dot1, dot2)
142
+
143
+ while True:
144
+ i = (i - 1) % len(polygon)
145
+ add_segment = Segment(polygon[i][0], polygon[(i + 1) % len(polygon)][0])
146
+ if add_segment.angle_ok_with(split_segment):
147
+ split_segment = Segment(add_segment.a, split_segment.b)
148
+ else:
149
+ break
150
+
151
+ while True:
152
+ j = (j + 1) % len(polygon)
153
+ add_segment = Segment(polygon[(j - 1) % len(polygon)][0], polygon[j][0])
154
+ if add_segment.angle_ok_with(split_segment):
155
+ split_segment = Segment(split_segment.a, add_segment.b)
156
+ else:
157
+ break
158
+
159
+ return split_segment
160
+
161
+ @staticmethod
162
+ def union_all(segments):
163
+ unioned_segments = True
164
+ while unioned_segments:
165
+ unioned_segments = False
166
+ dedup_segments = []
167
+ used = []
168
+ for i, s1 in enumerate(segments):
169
+ for s2 in segments[i + 1:]:
170
+ if s2 in used:
171
+ continue
172
+
173
+ s3 = s1.union(s2)
174
+ if s3 is not None:
175
+ unioned_segments = True
176
+ dedup_segments += [s3]
177
+ used.append(s1)
178
+ used.append(s2)
179
+ break
180
+
181
+ if s1 not in used:
182
+ dedup_segments += [s1]
183
+
184
+ segments = dedup_segments
185
+
186
+ return dedup_segments
187
+
188
+ def projected_point(self, p):
189
+ a = np.array(self.a)
190
+ b = np.array(self.b)
191
+ p = np.array(p)
192
+ ap = p - a
193
+ ab = b - a
194
+ if ab[0] == 0 and ab[1] == 0:
195
+ return a
196
+ result = a + np.dot(ap, ab) / np.dot(ab, ab) * ab
197
+ return (round(result[0]), round(result[1]))
requirements.txt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ opencv-python
2
+ requests
3
+ gradio
webui.bat ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ @echo off
2
+
3
+ :: The original source of the webui.bat file is stable-diffusion-webui
4
+ :: Modified and enhanced by Gemini with features for venv management and requirements handling.
5
+
6
+ :: --------- Configuration ---------
7
+ set COMMANDLINE_ARGS=
8
+ :: Define the name of the Launch application
9
+ set APPLICATION_NAME=app.py
10
+ :: Define the name of the virtual environment directory
11
+ set VENV_NAME=venv
12
+ :: Set to 1 to always attempt to update packages from requirements.txt on every launch
13
+ set ALWAYS_UPDATE_REQS=1
14
+ :: ---------------------------------
15
+
16
+
17
+ :: Set PYTHON executable if not already defined
18
+ if not defined PYTHON (set PYTHON=python)
19
+ :: Set VENV_DIR using VENV_NAME if not already defined
20
+ if not defined VENV_DIR (set "VENV_DIR=%~dp0%VENV_NAME%")
21
+
22
+ mkdir tmp 2>NUL
23
+
24
+ :: Check if Python is callable
25
+ %PYTHON% -c "" >tmp/stdout.txt 2>tmp/stderr.txt
26
+ if %ERRORLEVEL% == 0 goto :check_pip
27
+ echo Couldn't launch python
28
+ goto :show_stdout_stderr
29
+
30
+ :check_pip
31
+ :: Check if pip is available
32
+ %PYTHON% -mpip --help >tmp/stdout.txt 2>tmp/stderr.txt
33
+ if %ERRORLEVEL% == 0 goto :start_venv
34
+ :: If pip is not available and PIP_INSTALLER_LOCATION is set, try to install pip
35
+ if "%PIP_INSTALLER_LOCATION%" == "" goto :show_stdout_stderr
36
+ %PYTHON% "%PIP_INSTALLER_LOCATION%" >tmp/stdout.txt 2>tmp/stderr.txt
37
+ if %ERRORLEVEL% == 0 goto :start_venv
38
+ echo Couldn't install pip
39
+ goto :show_stdout_stderr
40
+
41
+ :start_venv
42
+ :: Skip venv creation/activation if VENV_DIR is explicitly set to "-"
43
+ if ["%VENV_DIR%"] == ["-"] goto :skip_venv_entirely
44
+ :: Skip venv creation/activation if SKIP_VENV is set to "1"
45
+ if ["%SKIP_VENV%"] == ["1"] goto :skip_venv_entirely
46
+
47
+ :: Check if the venv already exists by looking for Python.exe in its Scripts directory
48
+ dir "%VENV_DIR%\Scripts\Python.exe" >tmp/stdout.txt 2>tmp/stderr.txt
49
+ if %ERRORLEVEL% == 0 goto :activate_venv_and_maybe_update
50
+
51
+ :: Venv does not exist, create it
52
+ echo Virtual environment not found in "%VENV_DIR%". Creating a new one.
53
+ for /f "delims=" %%i in ('CALL %PYTHON% -c "import sys; print(sys.executable)"') do set PYTHON_FULLNAME="%%i"
54
+ echo Creating venv in directory %VENV_DIR% using python %PYTHON_FULLNAME%
55
+ %PYTHON_FULLNAME% -m venv "%VENV_DIR%" >tmp/stdout.txt 2>tmp/stderr.txt
56
+ if %ERRORLEVEL% NEQ 0 (
57
+ echo Unable to create venv in directory "%VENV_DIR%"
58
+ goto :show_stdout_stderr
59
+ )
60
+ echo Venv created.
61
+
62
+ :: Install requirements for the first time if venv was just created
63
+ :: This section handles the initial installation of packages from requirements.txt
64
+ :: immediately after a new virtual environment is created.
65
+ echo Checking for requirements.txt for initial setup in %~dp0
66
+ if exist "%~dp0requirements.txt" (
67
+ echo Found requirements.txt, attempting to install for initial setup...
68
+ call "%VENV_DIR%\Scripts\activate.bat"
69
+ echo Installing packages from requirements.txt ^(initial setup^)...
70
+ "%VENV_DIR%\Scripts\python.exe" -m pip install -r "%~dp0requirements.txt"
71
+ if %ERRORLEVEL% NEQ 0 (
72
+ echo Failed to install requirements during initial setup. Please check the output above.
73
+ pause
74
+ goto :show_stdout_stderr_custom_pip_initial
75
+ )
76
+ echo Initial requirements installed successfully.
77
+ call "%VENV_DIR%\Scripts\deactivate.bat"
78
+ ) else (
79
+ echo No requirements.txt found for initial setup, skipping package installation.
80
+ )
81
+ goto :activate_venv_and_maybe_update
82
+
83
+
84
+ :activate_venv_and_maybe_update
85
+ :: This label is reached if the venv exists or was just created.
86
+ :: Set PYTHON to point to the venv's Python interpreter.
87
+ set PYTHON="%VENV_DIR%\Scripts\Python.exe"
88
+ echo Activating venv: %PYTHON%
89
+
90
+ :: Always update requirements if ALWAYS_UPDATE_REQS is 1
91
+ :: This section allows for updating packages from requirements.txt on every launch
92
+ :: if the ALWAYS_UPDATE_REQS variable is set to 1.
93
+ if defined ALWAYS_UPDATE_REQS (
94
+ if "%ALWAYS_UPDATE_REQS%"=="1" (
95
+ echo ALWAYS_UPDATE_REQS is enabled.
96
+ if exist "%~dp0requirements.txt" (
97
+ echo Attempting to update packages from requirements.txt...
98
+ REM No need to call activate.bat here again, PYTHON is already set to the venv's python
99
+ %PYTHON% -m pip install -r "%~dp0requirements.txt"
100
+ if %ERRORLEVEL% NEQ 0 (
101
+ echo Failed to update requirements. Please check the output above.
102
+ pause
103
+ goto :endofscript
104
+ )
105
+ echo Requirements updated successfully.
106
+ ) else (
107
+ echo ALWAYS_UPDATE_REQS is enabled, but no requirements.txt found. Skipping update.
108
+ )
109
+ ) else (
110
+ echo ALWAYS_UPDATE_REQS is not enabled or not set to 1. Skipping routine update.
111
+ )
112
+ )
113
+
114
+ goto :launch
115
+
116
+ :skip_venv_entirely
117
+ :: This label is reached if venv usage is explicitly skipped.
118
+ echo Skipping venv.
119
+ goto :launch
120
+
121
+ :launch
122
+ :: Launch the main application
123
+ echo Launching Web UI with arguments: %COMMANDLINE_ARGS% %*
124
+ %PYTHON% %APPLICATION_NAME% %COMMANDLINE_ARGS% %*
125
+ echo Launch finished.
126
+ pause
127
+ exit /b
128
+
129
+ :show_stdout_stderr_custom_pip_initial
130
+ :: Custom error handler for failures during the initial pip install process.
131
+ echo.
132
+ echo exit code ^(pip initial install^): %errorlevel%
133
+ echo Errors during initial pip install. See output above.
134
+ echo.
135
+ echo Launch unsuccessful. Exiting.
136
+ pause
137
+ exit /b
138
+
139
+
140
+ :show_stdout_stderr
141
+ :: General error handler: displays stdout and stderr from the tmp directory.
142
+ echo.
143
+ echo exit code: %errorlevel%
144
+
145
+ for /f %%i in ("tmp\stdout.txt") do set size=%%~zi
146
+ if %size% equ 0 goto :show_stderr
147
+ echo.
148
+ echo stdout:
149
+ type tmp\stdout.txt
150
+
151
+ :show_stderr
152
+ for /f %%i in ("tmp\stderr.txt") do set size=%%~zi
153
+ if %size% equ 0 goto :endofscript
154
+ echo.
155
+ echo stderr:
156
+ type tmp\stderr.txt
157
+
158
+ :endofscript
159
+ echo.
160
+ echo Launch unsuccessful. Exiting.
161
+ pause
162
+ exit /b