WHG2023 commited on
Commit
4252b44
·
1 Parent(s): 87eec00

Enhanced Dual Output Features - Both descriptions AND LaTeX code + Multiple compilation options + Professional USPTO-ready vector graphics + Immediate understanding without compilation barriers

Browse files
.env ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ GRADIO_API_KEY=your_gradio_api_key
2
+ MODAL_API_KEY=ak-MJz2rp4PN5L0wrja0ZU6C9
3
+ NEBIUS_API_KEY=eyJhbGciOiJIUzI1NiIsImtpZCI6IlV6SXJWd1h0dnprLVRvdzlLZWstc0M1akptWXBvX1VaVkxUZlpnMDRlOFUiLCJ0eXAiOiJKV1QifQ.eyJzdWIiOiJnb29nbGUtb2F1dGgyfDEwMTg3NzczODM4MjkyNTg5OTYyNSIsInNjb3BlIjoib3BlbmlkIG9mZmxpbmVfYWNjZXNzIiwiaXNzIjoiYXBpX2tleV9pc3N1ZXIiLCJhdWQiOlsiaHR0cHM6Ly9uZWJpdXMtaW5mZXJlbmNlLmV1LmF1dGgwLmNvbS9hcGkvdjIvIl0sImV4cCI6MTkwNzM1Nzk5MywidXVpZCI6ImVmNGMyYmI1LTU0ZWEtNDg3Yi04Mjc1LWNjZTU0NTlhMWJlMyIsIm5hbWUiOiJIYWNrMjAyNSIsImV4cGlyZXNfYXQiOiIyMDMwLTA2LTEwVDIxOjM5OjUzKzAwMDAifQ.jgNqVFRs3y88YVkTxTUkwtA0NXCBxtyTBz7mePnNbPM
4
+ ANTHROPIC_API_KEY=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
5
+ OPENAI_API_KEY=your_openai_api_key
6
+ HYPERBOLIC_API_KEY=your_hyperbolic_api_key
7
+ MISTRAL_API_KEY=your_mistral_api_key
8
+ SAMBANOVA_API_KEY=your_sambanova_api_key
9
+ OPENROUTER_API_KEY=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
10
+ PUBMED_API_KEY=c0c9587b298469a200bdd7f28bd0d4b02b08
11
+ MODAL_BACKEND_URL="https://gebhardt-wolfh--patent-architect-fastapi-app.modal.run"
12
+ GROQ_API_KEY="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"
13
+ TAVILY_API_KEY="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"
14
+ GEMINI_API_KEY="AIzaSyCQU-TsvYphz_gvJuOq2IagCsPOmE3_kTM"
ENHANCED_DUAL_OUTPUT_FEATURES.md ADDED
@@ -0,0 +1,356 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # 🚀 Enhanced Dual Output Features for Patent Architect AI v2
2
+
3
+ ## 🌟 Overview
4
+
5
+ Patent Architect AI v2 now features **Enhanced Dual Output** - a revolutionary approach that provides both detailed technical descriptions AND professional LaTeX/TikZ code for patent figures, with multiple compilation options including embedded web-based conversion.
6
+
7
+ ## ✨ What's New
8
+
9
+ ### **Before Enhancement:**
10
+ - ❌ Either text descriptions OR LaTeX code
11
+ - ❌ Limited viewing options
12
+ - ❌ Manual compilation required
13
+ - ❌ No immediate understanding
14
+
15
+ ### **After Enhancement:**
16
+ - ✅ **Both detailed descriptions AND LaTeX code**
17
+ - ✅ **Multiple compilation options** (local, web-based, Overleaf)
18
+ - ✅ **Pre-compiled images** when possible
19
+ - ✅ **Immediate understanding** + professional quality
20
+ - ✅ **USPTO-ready output** with vector graphics
21
+
22
+ ---
23
+
24
+ ## 🎯 Key Features
25
+
26
+ ### 1. **Dual Output Generation**
27
+ ```python
28
+ result = generator.generate_dual_output_figure(
29
+ invention_description="Smart coffee mug...",
30
+ figure_type="technical_diagram"
31
+ )
32
+
33
+ # Returns:
34
+ # - Detailed technical description (readable immediately)
35
+ # - Professional LaTeX/TikZ code (compilation-ready)
36
+ # - Optional pre-compiled image
37
+ ```
38
+
39
+ ### 2. **Multiple Compilation Options**
40
+
41
+ #### **Option A: Pre-Compiled Images** 🖼️
42
+ - Automatic compilation when LaTeX tools available
43
+ - Both local and web-based compilation attempts
44
+ - Immediate viewing without additional steps
45
+
46
+ #### **Option B: Overleaf Integration** 🌐 (Recommended)
47
+ - Direct upload to Overleaf.com (free account)
48
+ - One-click compilation to professional PDF
49
+ - No software installation required
50
+
51
+ #### **Option C: Local Compilation** 💻
52
+ - Full LaTeX installation with TikZ package
53
+ - Command-line compilation for developers
54
+ - Maximum control and customization
55
+
56
+ #### **Option D: Web Preview** 📱
57
+ - LaTeX Live for quick previews
58
+ - Mobile-friendly viewing
59
+ - Instant code validation
60
+
61
+ ### 3. **Enhanced User Experience**
62
+
63
+ #### **Immediate Understanding:**
64
+ ```
65
+ 📝 Technical Description Preview:
66
+ The smart coffee mug comprises a cylindrical vessel (1) with
67
+ an integrated phase-change material chamber (2) positioned
68
+ between the inner wall (3) and outer wall (4)...
69
+ ```
70
+
71
+ #### **Professional LaTeX Code:**
72
+ ```latex
73
+ \documentclass[12pt]{article}
74
+ \usepackage{tikz}
75
+ \begin{tikzpicture}[scale=1.2]
76
+ \draw[thick] (0,0) rectangle (4,6);
77
+ \node at (2,5.5) {1};
78
+ \draw[fill=blue!20] (0.5,1) rectangle (3.5,4);
79
+ \node at (2,2.5) {2};
80
+ ...
81
+ \end{tikzpicture}
82
+ ```
83
+
84
+ ---
85
+
86
+ ## 📊 Enhanced Integration with Patent Architect
87
+
88
+ ### **New Output Format:**
89
+
90
+ ```markdown
91
+ ## 🎨 Enhanced Patent Figures with Dual Output
92
+
93
+ ✨ Generated 3 complete figure sets with both detailed descriptions AND LaTeX/TikZ code
94
+ 🖼️ 2/3 figures successfully compiled to images (1 local, 1 web compilation)
95
+
96
+ ### Figure 1: Overall System Diagram
97
+
98
+ 🖼️ Compiled Image: `patent_figure_technical_diagram_20241215_143022.png` (Local compilation)
99
+
100
+ 📝 Technical Description:
101
+ The smart coffee mug system comprises a primary vessel with integrated temperature
102
+ control using phase-change materials...
103
+
104
+ 🎨 LaTeX/TikZ Code: `patent_figure_technical_diagram_20241215_143022.tex`
105
+
106
+ 🚀 Multiple Viewing Options:
107
+
108
+ 1. ✅ Pre-Compiled Image Available: View in output folder
109
+ 2. 🌐 Online Compilation (Easiest):
110
+ - Go to Overleaf.com (free account)
111
+ - Upload .tex file → Click 'Recompile'
112
+ 3. 💻 Local Compilation: pdflatex filename.tex
113
+ 4. 📱 Mobile/Quick View: LaTeX Live (latexlive.com)
114
+ ```
115
+
116
+ ### **Business Benefits:**
117
+
118
+ - **Immediate Understanding:** Users read descriptions right away
119
+ - **Professional Quality:** LaTeX creates publication-grade vector graphics
120
+ - **USPTO Preferred:** Vector graphics ideal for patent applications
121
+ - **Multiple Skill Levels:** Options for beginners and experts
122
+ - **Cost Effective:** No expensive software required
123
+
124
+ ---
125
+
126
+ ## 🔧 Technical Implementation
127
+
128
+ ### **Web-Based LaTeX Compilation**
129
+
130
+ ```python
131
+ def _compile_latex_via_web_api(self, latex_code: str, filename: str) -> Optional[str]:
132
+ """Compile LaTeX using web-based APIs."""
133
+ api_url = "https://latex.vercel.app/api/v2"
134
+
135
+ payload = {
136
+ "code": latex_code,
137
+ "format": "png",
138
+ "quality": 300
139
+ }
140
+
141
+ response = requests.post(api_url, json=payload, timeout=30)
142
+ # Returns compiled PNG image
143
+ ```
144
+
145
+ ### **Enhanced Local Compilation**
146
+
147
+ ```python
148
+ def _compile_latex_to_image(self, latex_filepath: str) -> Optional[str]:
149
+ """Enhanced compilation with multiple fallback methods."""
150
+
151
+ # Try local compilation first
152
+ if self.latex_available:
153
+ # pdflatex compilation
154
+ # Multiple PDF→PNG conversion methods
155
+
156
+ # Fallback to web compilation
157
+ web_result = self._compile_latex_via_web_api(latex_code, filename)
158
+ return web_result
159
+ ```
160
+
161
+ ### **Dual Output Generation**
162
+
163
+ ```python
164
+ def generate_dual_output_figure(self, invention_description: str) -> Dict:
165
+ """Generate both description AND LaTeX code."""
166
+
167
+ # Step 1: Generate detailed text description
168
+ text_response = self.model.generate_content([text_prompt])
169
+ detailed_description = text_response.text
170
+
171
+ # Step 2: Generate LaTeX/TikZ code
172
+ latex_response = self.model.generate_content([latex_prompt])
173
+ latex_code = latex_response.text
174
+
175
+ # Step 3: Try compilation
176
+ compiled_image_path = self._compile_latex_to_image(latex_filepath)
177
+
178
+ return {
179
+ "text_description": detailed_description,
180
+ "latex_code": latex_code,
181
+ "compiled_image_path": compiled_image_path,
182
+ # ... enhanced metadata
183
+ }
184
+ ```
185
+
186
+ ---
187
+
188
+ ## 🎯 Usage Examples
189
+
190
+ ### **Example 1: Basic Dual Output**
191
+
192
+ ```python
193
+ from gemini_image_generator import GeminiImageGenerator
194
+
195
+ generator = GeminiImageGenerator()
196
+
197
+ result = generator.generate_dual_output_figure(
198
+ "A smart coffee mug with temperature control",
199
+ "technical_diagram"
200
+ )
201
+
202
+ if result["success"]:
203
+ print("📝 Description:", result["text_description"])
204
+ print("🎨 LaTeX Code:", result["latex_code"][:200] + "...")
205
+ if result["compiled_image_path"]:
206
+ print("🖼️ Image:", result["compiled_image_path"])
207
+ ```
208
+
209
+ ### **Example 2: Patent Architect Integration**
210
+
211
+ ```python
212
+ # This happens automatically in Patent Architect
213
+ integration_result = generator.integrate_with_patent_architect(
214
+ "Modular vertical farming system with adaptive LED lighting"
215
+ )
216
+
217
+ # Returns enhanced content with:
218
+ # - Multiple figure descriptions
219
+ # - Multiple LaTeX code sets
220
+ # - Compilation statistics
221
+ # - User guidance
222
+ ```
223
+
224
+ ### **Example 3: Custom Compilation**
225
+
226
+ ```python
227
+ # Generate without compilation
228
+ result = generator.generate_dual_output_figure(invention, figure_type)
229
+
230
+ # Then compile manually with different options
231
+ if result["latex_code"]:
232
+ # Option 1: Try web compilation
233
+ web_image = generator._compile_latex_via_web_api(
234
+ result["latex_code"],
235
+ "custom_figure"
236
+ )
237
+
238
+ # Option 2: Save for Overleaf
239
+ with open("for_overleaf.tex", "w") as f:
240
+ f.write(result["latex_code"])
241
+ ```
242
+
243
+ ---
244
+
245
+ ## 📋 File Structure
246
+
247
+ ### **Enhanced Output Files:**
248
+
249
+ ```
250
+ patent_architect_figures/
251
+ ├── patent_figure_technical_diagram_20241215_143022_description.txt
252
+ ├── patent_figure_technical_diagram_20241215_143022.tex
253
+ ├── patent_figure_technical_diagram_20241215_143022.png (if compiled)
254
+ ├── patent_figure_cross_section_20241215_143045_description.txt
255
+ ├── patent_figure_cross_section_20241215_143045.tex
256
+ └── patent_figure_cross_section_20241215_143045.pdf (if compiled)
257
+ ```
258
+
259
+ ### **File Categories:**
260
+ - **📝 Description Files:** `*_description.txt` - Immediate reading
261
+ - **🎨 LaTeX Files:** `*.tex` - Professional compilation
262
+ - **🖼️ Image Files:** `*.png`, `*.pdf` - Ready-to-use visuals
263
+
264
+ ---
265
+
266
+ ## 🏆 Advantages Over Traditional Approaches
267
+
268
+ ### **Compared to Image-Only Generation:**
269
+ - ✅ **Immediate understanding** without compilation
270
+ - ✅ **Editable vector graphics** instead of fixed pixels
271
+ - ✅ **Multiple viewing options** for different skill levels
272
+ - ✅ **Professional quality** suitable for USPTO submission
273
+
274
+ ### **Compared to LaTeX-Only Generation:**
275
+ - ✅ **No technical barrier** - read descriptions immediately
276
+ - ✅ **User-friendly** for non-LaTeX users
277
+ - ✅ **Multiple compilation paths** including web-based
278
+ - ✅ **Gradual learning** from descriptions to code
279
+
280
+ ### **Compared to Description-Only Generation:**
281
+ - ✅ **Professional output** ready for patent submission
282
+ - ✅ **Vector graphics** with infinite scaling
283
+ - ✅ **Industry standard** LaTeX format
284
+ - ✅ **Customizable** for specific requirements
285
+
286
+ ---
287
+
288
+ ## 🚀 Getting Started
289
+
290
+ ### **1. Quick Test:**
291
+ ```bash
292
+ python test_latex_visualization.py
293
+ ```
294
+
295
+ ### **2. Patent Architect Integration:**
296
+ ```bash
297
+ python app.py
298
+ # Then input: "Smart coffee mug with temperature control"
299
+ ```
300
+
301
+ ### **3. Direct API Usage:**
302
+ ```python
303
+ from gemini_image_generator import GeminiImageGenerator
304
+
305
+ generator = GeminiImageGenerator()
306
+ result = generator.integrate_with_patent_architect("Your invention here")
307
+ print(result["content"])
308
+ ```
309
+
310
+ ---
311
+
312
+ ## 🎯 Best Practices
313
+
314
+ ### **For Immediate Use:**
315
+ 1. **Read the generated descriptions** for instant understanding
316
+ 2. **Check for pre-compiled images** in the output folder
317
+ 3. **Use descriptions for patent applications** when images aren't needed
318
+
319
+ ### **For Professional Quality:**
320
+ 1. **Upload LaTeX to Overleaf.com** for best results
321
+ 2. **Compile to PDF** for USPTO submission
322
+ 3. **Edit LaTeX code** for custom adjustments
323
+
324
+ ### **For Developers:**
325
+ 1. **Install local LaTeX** for fastest compilation
326
+ 2. **Use command-line tools** for batch processing
327
+ 3. **Modify generation prompts** for specific requirements
328
+
329
+ ---
330
+
331
+ ## 🔮 Future Enhancements
332
+
333
+ ### **Planned Features:**
334
+ - 🔄 **Interactive editing** of LaTeX code in browser
335
+ - 🎨 **Style templates** for different patent offices
336
+ - 📱 **Mobile app** for viewing and compilation
337
+ - 🤝 **Team collaboration** features
338
+ - 📊 **Analytics** on compilation success rates
339
+
340
+ ### **API Improvements:**
341
+ - 🌐 **More web compilation services** for redundancy
342
+ - ⚡ **Faster compilation** with optimized APIs
343
+ - 🎯 **Custom templates** for specific industries
344
+ - 🔧 **Debugging tools** for LaTeX issues
345
+
346
+ ---
347
+
348
+ ## 🎉 Conclusion
349
+
350
+ The Enhanced Dual Output feature transforms Patent Architect AI v2 into a comprehensive patent figure generation platform that serves both technical and non-technical users. By providing both immediate understanding through descriptions AND professional quality through LaTeX code, with multiple compilation options, we've created the most user-friendly yet powerful patent figure system available.
351
+
352
+ **The result:** Users get immediate value while maintaining access to professional-grade output suitable for USPTO submission.
353
+
354
+ ---
355
+
356
+ *Patent Architect AI v2 - Where Innovation Meets Intelligence* 🏆
__pycache__/gemini_image_generator.cpython-312.pyc ADDED
Binary file (30.2 kB). View file
 
__pycache__/real_ai_agents_implementation.cpython-312.pyc ADDED
Binary file (25 kB). View file
 
app.py CHANGED
@@ -352,23 +352,37 @@ def format_patent_section(agent_name: str, content: str, thought: str = None, so
352
  # COMPLETELY REPLACE the backend content with Gemini-generated content
353
  content = gemini_result["content"]
354
 
355
- # Check if we have actual images or LaTeX code
356
- if gemini_result.get("images_generated", 0) > 0:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
357
  if gemini_result.get("latex_generated", 0) > 0:
358
  content += "\n\n*🎨 Generated LaTeX/TikZ code using Google Gemini 2.0 Flash - Compile with pdflatex to create professional patent figures*\n\n"
359
  content += f"*💡 To view images: Compile the .tex files in `{os.path.expanduser('~')}/patent_architect_figures/`*\n\n"
 
360
  else:
361
  content += "\n\n*🎨 Generated visual patent figures using Google Gemini 2.0 Flash - Professional images ready for USPTO submission*\n\n"
 
362
  else:
363
  content += "\n\n*📝 Generated detailed patent figure descriptions using Google Gemini 2.0 Flash - Professional technical specifications*\n\n"
 
364
 
365
  print("✅ Gemini image generation successful - Backend content replaced")
366
 
367
- # Update thought to reflect Gemini generation
368
- if gemini_result.get("latex_generated", 0) > 0:
369
- thought = "Generated professional LaTeX/TikZ code for patent figures using Google Gemini 2.0 Flash. Code can be compiled to create high-quality technical drawings with numbered components and USPTO-ready formatting."
370
- else:
371
- thought = "Generated high-quality patent figures using Google Gemini 2.0 Flash with professional technical drawings, numbered components, and USPTO-ready formatting."
372
  else:
373
  print(f"❌ Gemini generation failed: {gemini_result.get('error', 'Unknown error')}")
374
 
 
352
  # COMPLETELY REPLACE the backend content with Gemini-generated content
353
  content = gemini_result["content"]
354
 
355
+ # Check if we have dual output (descriptions + LaTeX)
356
+ if gemini_result.get("dual_output"):
357
+ # Enhanced dual output available
358
+ descriptions_available = gemini_result.get("descriptions_available", 0)
359
+ latex_available = gemini_result.get("latex_code_available", 0)
360
+ compiled_images = gemini_result.get("compiled_images", 0)
361
+
362
+ content += f"\n\n*🚀 Enhanced Dual Output: {descriptions_available} technical descriptions + {latex_available} LaTeX/TikZ code sets generated*\n\n"
363
+
364
+ if compiled_images > 0:
365
+ content += f"*🖼️ {compiled_images} figures successfully compiled to images - View immediately in output folder*\n\n"
366
+
367
+ content += f"*💡 Best Experience: Read descriptions above, then use LaTeX code with Overleaf.com for professional figures*\n\n"
368
+
369
+ # Update thought to reflect dual output
370
+ thought = f"Generated comprehensive dual output with {descriptions_available} detailed technical descriptions AND {latex_available} professional LaTeX/TikZ code sets. Users can read descriptions immediately and compile LaTeX for USPTO-ready vector graphics."
371
+
372
+ elif gemini_result.get("images_generated", 0) > 0:
373
  if gemini_result.get("latex_generated", 0) > 0:
374
  content += "\n\n*🎨 Generated LaTeX/TikZ code using Google Gemini 2.0 Flash - Compile with pdflatex to create professional patent figures*\n\n"
375
  content += f"*💡 To view images: Compile the .tex files in `{os.path.expanduser('~')}/patent_architect_figures/`*\n\n"
376
+ thought = "Generated professional LaTeX/TikZ code for patent figures using Google Gemini 2.0 Flash. Code can be compiled to create high-quality technical drawings with numbered components and USPTO-ready formatting."
377
  else:
378
  content += "\n\n*🎨 Generated visual patent figures using Google Gemini 2.0 Flash - Professional images ready for USPTO submission*\n\n"
379
+ thought = "Generated high-quality patent figures using Google Gemini 2.0 Flash with professional technical drawings, numbered components, and USPTO-ready formatting."
380
  else:
381
  content += "\n\n*📝 Generated detailed patent figure descriptions using Google Gemini 2.0 Flash - Professional technical specifications*\n\n"
382
+ thought = "Generated detailed technical specifications for patent figures using Google Gemini 2.0 Flash with professional patent formatting."
383
 
384
  print("✅ Gemini image generation successful - Backend content replaced")
385
 
 
 
 
 
 
386
  else:
387
  print(f"❌ Gemini generation failed: {gemini_result.get('error', 'Unknown error')}")
388
 
gemini_image_generator.py CHANGED
@@ -1,7 +1,9 @@
1
  #!/usr/bin/env python3
2
  """
3
- Gemini Image Generator Integration for Patent Architect AI v2
4
- Provides high-quality technical figure generation using Google's Gemini 2.0 Flash model.
 
 
5
  """
6
 
7
  import os
@@ -9,6 +11,8 @@ import json
9
  import base64
10
  import requests
11
  import tempfile
 
 
12
  from typing import Dict, List, Optional, Tuple
13
  import google.generativeai as genai
14
  from datetime import datetime
@@ -18,10 +22,10 @@ from dotenv import load_dotenv
18
  load_dotenv()
19
 
20
  class GeminiImageGenerator:
21
- """Integrates Google Gemini 2.0 Flash for patent figure generation."""
22
 
23
  def __init__(self, api_key: Optional[str] = None, output_dir: Optional[str] = None):
24
- """Initialize the Gemini image generator.
25
 
26
  Args:
27
  api_key: Google Gemini API key
@@ -39,6 +43,9 @@ class GeminiImageGenerator:
39
  self.output_dir = output_dir or os.path.join(os.path.expanduser("~"), "patent_architect_figures")
40
  os.makedirs(self.output_dir, exist_ok=True)
41
 
 
 
 
42
  # Image generation parameters
43
  self.generation_config = {
44
  'temperature': 0.4,
@@ -47,202 +54,242 @@ class GeminiImageGenerator:
47
  'max_output_tokens': 8192,
48
  }
49
 
50
- def generate_patent_figure(self,
51
- invention_description: str,
52
- figure_type: str = "technical_diagram",
53
- specific_requirements: Optional[str] = None) -> Dict:
54
- """Generate a patent-style technical figure.
 
 
 
 
 
 
 
 
55
 
56
  Args:
57
- invention_description: Description of the invention
58
- figure_type: Type of figure (technical_diagram, cross_section, system_overview, etc.)
59
- specific_requirements: Additional requirements for the figure
60
 
61
  Returns:
62
- Dict containing image data, file path, and metadata
63
  """
64
  try:
65
- # Create patent-specific prompt for image generation
66
- image_prompt = self._create_image_generation_prompt(
67
- invention_description,
68
- figure_type,
69
- specific_requirements
70
- )
71
 
72
- # Generate filename
73
- timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
 
 
 
74
 
75
- try:
76
- # Try to generate actual image using Gemini 2.0 Flash
77
- print(f"🎨 Attempting to generate actual image for {figure_type}...")
 
 
 
 
78
 
79
- # Configure for image generation
80
- image_generation_config = {
81
- 'temperature': 0.1, # Low temperature for consistent technical drawings
82
- 'top_k': 1,
83
- 'top_p': 0.8,
84
- 'max_output_tokens': 4096,
85
- }
86
 
87
- # Create image generation request
88
- response = self.model.generate_content([image_prompt], generation_config=image_generation_config)
 
 
 
89
 
90
- # Check if response contains image data
91
- if hasattr(response, 'parts') and any(hasattr(part, 'inline_data') for part in response.parts):
92
- # Extract image data
93
- for part in response.parts:
94
- if hasattr(part, 'inline_data'):
95
- image_data = part.inline_data.data
96
- mime_type = part.inline_data.mime_type
97
-
98
- # Handle actual image files
99
- if mime_type and 'image' in mime_type:
100
- extension = mime_type.split('/')[-1]
101
- image_filename = f"patent_figure_{figure_type}_{timestamp}.{extension}"
102
- image_filepath = os.path.join(self.output_dir, image_filename)
103
-
104
- # Decode and save image
105
- image_bytes = base64.b64decode(image_data)
106
- with open(image_filepath, 'wb') as f:
107
- f.write(image_bytes)
108
-
109
- print(f"✅ Successfully generated image: {image_filename}")
110
-
111
- return {
112
- "success": True,
113
- "image_generated": True,
114
- "image_path": image_filepath,
115
- "figure_description": f"Generated technical patent figure for {figure_type}",
116
- "file_path": image_filepath,
117
- "figure_type": figure_type,
118
- "timestamp": timestamp,
119
- "mime_type": mime_type,
120
- "metadata": {
121
- "invention": invention_description,
122
- "requirements": specific_requirements,
123
- "generation_config": image_generation_config
124
- }
125
- }
126
-
127
- # Handle LaTeX/TikZ code generation (common for technical drawings)
128
- elif image_data:
129
- try:
130
- # Decode the data as text (might be LaTeX/TikZ code)
131
- code_content = base64.b64decode(image_data).decode('utf-8')
132
-
133
- # Check if it looks like LaTeX/TikZ code
134
- if any(keyword in code_content.lower() for keyword in ['tikz', 'latex', 'documentclass', '\\draw']):
135
- print(f"🎨 Generated LaTeX/TikZ code for technical drawing")
136
-
137
- # Save LaTeX code
138
- latex_filename = f"patent_figure_{figure_type}_{timestamp}.tex"
139
- latex_filepath = os.path.join(self.output_dir, latex_filename)
140
-
141
- with open(latex_filepath, 'w', encoding='utf-8') as f:
142
- f.write(f"% Patent Figure - {figure_type}\n")
143
- f.write(f"% Generated: {datetime.now().isoformat()}\n")
144
- f.write(f"% Invention: {invention_description}\n\n")
145
- f.write(code_content)
146
-
147
- return {
148
- "success": True,
149
- "image_generated": True,
150
- "latex_generated": True,
151
- "image_path": latex_filepath,
152
- "figure_description": f"Generated LaTeX/TikZ code for technical patent figure: {figure_type}",
153
- "file_path": latex_filepath,
154
- "figure_type": figure_type,
155
- "timestamp": timestamp,
156
- "mime_type": "application/x-latex",
157
- "code_preview": code_content[:200] + "..." if len(code_content) > 200 else code_content,
158
- "metadata": {
159
- "invention": invention_description,
160
- "requirements": specific_requirements,
161
- "generation_config": image_generation_config,
162
- "format": "LaTeX/TikZ"
163
- }
164
- }
165
- except Exception as decode_error:
166
- print(f"⚠️ Could not decode inline data as text: {decode_error}")
167
 
168
- # Check if response has text that might contain drawing code
169
- if hasattr(response, 'text') and response.text:
170
- response_text = response.text
171
-
172
- # Check if response contains LaTeX/TikZ code
173
- if any(keyword in response_text.lower() for keyword in ['tikz', 'latex', 'documentclass', '\\draw']):
174
- print(f"🎨 Found LaTeX/TikZ code in text response")
 
 
 
 
 
 
 
 
 
175
 
176
- # Save LaTeX code
177
- latex_filename = f"patent_figure_{figure_type}_{timestamp}.tex"
178
- latex_filepath = os.path.join(self.output_dir, latex_filename)
 
 
 
 
 
 
179
 
180
- with open(latex_filepath, 'w', encoding='utf-8') as f:
181
- f.write(f"% Patent Figure - {figure_type}\n")
182
- f.write(f"% Generated: {datetime.now().isoformat()}\n")
183
- f.write(f"% Invention: {invention_description}\n\n")
184
- f.write(response_text)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
185
 
186
- return {
187
- "success": True,
188
- "image_generated": True,
189
- "latex_generated": True,
190
- "image_path": latex_filepath,
191
- "figure_description": f"Generated LaTeX/TikZ code for technical patent figure: {figure_type}",
192
- "file_path": latex_filepath,
193
- "figure_type": figure_type,
194
- "timestamp": timestamp,
195
- "mime_type": "application/x-latex",
196
- "code_preview": response_text[:300] + "..." if len(response_text) > 300 else response_text,
197
- "metadata": {
198
- "invention": invention_description,
199
- "requirements": specific_requirements,
200
- "generation_config": image_generation_config,
201
- "format": "LaTeX/TikZ"
202
- }
203
- }
204
-
205
- # If no image data, fall back to text description generation
206
- print(f"ℹ️ No image data in response, generating detailed text description...")
207
-
208
- except Exception as image_error:
209
- print(f"⚠️ Image generation failed ({str(image_error)}), falling back to text description...")
210
 
211
- # Fallback: Generate detailed text description
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
212
  text_prompt = self._create_patent_figure_prompt(
213
  invention_description,
214
  figure_type,
215
  specific_requirements
216
  )
217
 
218
- response = self.model.generate_content([text_prompt])
219
- figure_description = response.text
 
 
 
 
 
 
 
 
 
 
 
 
220
 
221
- # Save text description
222
- filename = f"patent_figure_{figure_type}_{timestamp}.txt"
223
- filepath = os.path.join(self.output_dir, filename)
224
 
225
- with open(filepath, 'w', encoding='utf-8') as f:
 
 
 
 
226
  f.write(f"Patent Figure Description\n")
227
  f.write(f"========================\n\n")
228
  f.write(f"Invention: {invention_description}\n\n")
229
  f.write(f"Figure Type: {figure_type}\n\n")
230
- f.write(f"Description:\n{figure_description}\n\n")
231
  f.write(f"Generated: {datetime.now().isoformat()}\n")
232
 
233
- print(f"✅ Generated detailed text description: {filename}")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
234
 
235
  return {
236
  "success": True,
237
- "image_generated": False,
238
- "figure_description": figure_description,
239
- "file_path": filepath,
 
 
 
 
240
  "figure_type": figure_type,
241
  "timestamp": timestamp,
 
 
242
  "metadata": {
243
  "invention": invention_description,
244
  "requirements": specific_requirements,
245
- "generation_config": self.generation_config
 
246
  }
247
  }
248
 
@@ -253,6 +300,15 @@ class GeminiImageGenerator:
253
  "figure_type": figure_type
254
  }
255
 
 
 
 
 
 
 
 
 
 
256
  def generate_multiple_figures(self,
257
  invention_description: str,
258
  figure_types: List[str] = None) -> List[Dict]:
@@ -346,7 +402,7 @@ a professional patent figure. Focus on technical accuracy and completeness."""
346
  invention_description: str,
347
  figure_type: str,
348
  specific_requirements: Optional[str] = None) -> str:
349
- """Create a prompt specifically for image generation.
350
 
351
  Args:
352
  invention_description: Description of the invention
@@ -354,29 +410,28 @@ a professional patent figure. Focus on technical accuracy and completeness."""
354
  specific_requirements: Additional requirements
355
 
356
  Returns:
357
- Optimized prompt for visual patent figure generation
358
  """
359
- base_prompt = f"""Create a professional patent-style technical drawing showing: {invention_description}
360
 
361
  Figure Type: {figure_type}
362
 
363
- VISUAL REQUIREMENTS:
364
- - Black and white line art technical drawing style
365
- - Clean, precise lines with consistent line weights
 
 
366
  - Numbered components (1, 2, 3, etc.) with clear labels
367
- - Professional patent figure formatting
368
- - No decorative elements - focus on technical clarity
369
- - Isometric or cross-sectional view as appropriate
370
- - Clear spatial relationships between components
371
- - Technical precision suitable for USPTO submission
372
 
373
- SPECIFIC DRAWING INSTRUCTIONS:
374
- - Use solid black lines on white background
375
- - Component numbers should be clearly visible
376
- - Include dimension lines where relevant
377
- - Show all essential components mentioned in the invention
378
- - Use cross-hatching for cross-sections
379
- - Include reference lines and callouts
380
 
381
  """
382
 
@@ -407,14 +462,14 @@ The drawing should be clear, precise, and technically accurate."""
407
  def integrate_with_patent_architect(self,
408
  invention_description: str,
409
  figure_descriptions: List[str] = None) -> Dict:
410
- """Integration point for Patent Architect AI v2 system.
411
 
412
  Args:
413
  invention_description: The invention being patented
414
  figure_descriptions: Specific figure descriptions needed
415
 
416
  Returns:
417
- Dict with content and image information for patent application
418
  """
419
  try:
420
  if figure_descriptions is None:
@@ -423,22 +478,26 @@ The drawing should be clear, precise, and technically accurate."""
423
  results = []
424
 
425
  for i, figure_type in enumerate(figure_types, 1):
426
- result = self.generate_patent_figure(invention_description, figure_type)
427
  if result["success"]:
428
  results.append({
429
  "figure_number": i,
430
- "description": result["figure_description"],
431
- "file_path": result["file_path"],
 
 
 
 
432
  "type": figure_type,
433
- "image_generated": result.get("image_generated", False),
434
- "image_path": result.get("image_path"),
435
- "mime_type": result.get("mime_type")
436
  })
437
  else:
438
  # Generate specific requested figures
439
  results = []
440
  for i, description in enumerate(figure_descriptions, 1):
441
- result = self.generate_patent_figure(
442
  invention_description,
443
  f"custom_figure_{i}",
444
  description
@@ -446,102 +505,135 @@ The drawing should be clear, precise, and technically accurate."""
446
  if result["success"]:
447
  results.append({
448
  "figure_number": i,
449
- "description": result["figure_description"],
450
- "file_path": result["file_path"],
 
 
 
 
451
  "type": "custom",
452
- "image_generated": result.get("image_generated", False),
453
- "image_path": result.get("image_path"),
454
- "mime_type": result.get("mime_type")
455
  })
456
 
457
- # Format for Patent Architect
458
  if results:
459
- content = "## Patent Figures\n\n"
460
 
461
- # Count actual images vs text descriptions
462
- images_generated = sum(1 for r in results if r.get("image_generated", False))
463
- latex_generated = sum(1 for r in results if r.get("latex_generated", False))
464
  total_figures = len(results)
 
 
 
465
 
466
- if images_generated > 0:
467
- if latex_generated > 0:
468
- content += f"*🎨 {images_generated}/{total_figures} figures generated as LaTeX/TikZ code using Google Gemini 2.0 Flash*\n\n"
469
- else:
470
- content += f"*🎨 {images_generated}/{total_figures} figures generated as actual images using Google Gemini 2.0 Flash*\n\n"
471
- else:
472
- content += f"*📝 {total_figures} figures generated as detailed technical descriptions (image generation in development)*\n\n"
 
 
 
 
 
473
 
474
  for figure in results:
475
  figure_title = figure['type'].replace('_', ' ').title()
476
  content += f"### Figure {figure['figure_number']}: {figure_title}\n\n"
477
 
478
- # If actual image was generated, show image info
479
- if figure.get("image_generated", False):
480
- if figure.get("latex_generated", False):
481
- # LaTeX/TikZ code generated
482
- latex_filename = os.path.basename(figure['image_path'])
483
- content += f"**🎨 Generated LaTeX/TikZ Code:** `{latex_filename}`\n\n"
484
- content += f"**Format:** Professional technical drawing code\n\n"
485
-
486
- # Show code preview if available
487
- if figure.get('code_preview'):
488
- content += f"**Code Preview:**\n```latex\n{figure['code_preview']}\n```\n\n"
489
-
490
- content += f"**Description:** {figure['description']}\n\n"
491
-
492
- # Enhanced viewing instructions
493
- content += f"**📋 To View This Figure:**\n"
494
- content += f"1. **Online (Easiest)**: Copy code to [Overleaf.com](https://overleaf.com) and compile\n"
495
- content += f"2. **Local**: Install LaTeX and run `pdflatex {latex_filename}`\n"
496
- content += f"3. **Result**: Professional vector graphics patent figure\n\n"
497
-
498
- content += f"*💡 LaTeX code creates publication-quality patent figures with numbered components*\n\n"
499
- content += f"*📁 File saved to: {figure['image_path']}*\n\n"
500
- else:
501
- # Actual image file generated
502
- image_filename = os.path.basename(figure['image_path'])
503
- content += f"**🖼️ Generated Image:** `{image_filename}`\n\n"
504
- content += f"**Image Type:** {figure.get('mime_type', 'Unknown')}\n\n"
505
- content += f"**Description:** {figure['description']}\n\n"
506
- content += f"*Image saved to: {figure['image_path']}*\n\n"
507
- else:
508
- # Show detailed text description
509
- content += f"{figure['description']}\n\n"
510
- content += f"*Description file: {os.path.basename(figure['file_path'])}*\n\n"
 
 
 
 
 
 
 
 
 
 
511
 
512
- # Add summary information
513
  content += "---\n\n"
514
- content += f"**Generation Summary:**\n"
515
- content += f"- Total Figures: {total_figures}\n"
516
- content += f"- Images Generated: {images_generated}\n"
517
- if latex_generated > 0:
518
- content += f"- LaTeX/TikZ Code: {latex_generated}\n"
519
- content += f"- Text Descriptions: {total_figures - images_generated}\n"
520
 
521
- if images_generated > 0:
522
- if latex_generated > 0:
523
- content += f"- Format: Professional LaTeX/TikZ technical drawing code\n"
524
- content += f"- USPTO Ready: Yes (compile LaTeX to generate images)\n"
525
- content += f"- Compilation: Use pdflatex with tikz package\n"
526
- else:
527
- content += f"- Image Format: Professional patent-style technical drawings\n"
528
- content += f"- USPTO Ready: Yes\n"
 
 
 
 
529
 
530
- content += f"- Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n\n"
531
 
532
  return {
533
  "success": True,
534
  "content": content,
535
  "figures": results,
536
  "total_figures": len(results),
537
- "images_generated": images_generated,
538
- "latex_generated": latex_generated,
539
- "text_descriptions": total_figures - images_generated
 
 
 
 
 
 
 
 
 
540
  }
541
  else:
542
  return {
543
  "success": False,
544
- "content": "### Technical Figures\n\nError generating patent figures. Please review invention description and try again.\n\n",
545
  "error": "No figures were successfully generated"
546
  }
547
 
@@ -585,7 +677,7 @@ def test_gemini_integration():
585
  print(f" 🎨 Format: {result.get('mime_type', 'Unknown')}")
586
  else:
587
  print(f"ℹ️ Generated detailed text description (image generation unavailable)")
588
- print(f" 📄 Description file: {result['file_path']}")
589
  else:
590
  print(f"❌ Individual generation failed: {result.get('error', 'Unknown error')}")
591
 
@@ -670,7 +762,7 @@ Draw this as a black and white technical patent figure."""
670
  print(f" 📎 Has inline data: ✅")
671
  print(f" 🎨 MIME type: {part.inline_data.mime_type}")
672
  else:
673
- print(f" 📎 Has inline data: ❌")
674
 
675
  if hasattr(response, 'text') and response.text:
676
  preview = response.text[:200] + "..." if len(response.text) > 200 else response.text
 
1
  #!/usr/bin/env python3
2
  """
3
+ Enhanced Gemini Image Generator Integration for Patent Architect AI v2
4
+ Provides high-quality technical figure generation with dual output:
5
+ - Detailed technical descriptions for immediate understanding
6
+ - Professional LaTeX/TikZ code with embedded compilation options
7
  """
8
 
9
  import os
 
11
  import base64
12
  import requests
13
  import tempfile
14
+ import subprocess
15
+ import shutil
16
  from typing import Dict, List, Optional, Tuple
17
  import google.generativeai as genai
18
  from datetime import datetime
 
22
  load_dotenv()
23
 
24
  class GeminiImageGenerator:
25
+ """Enhanced Gemini integration with dual output and embedded compilation."""
26
 
27
  def __init__(self, api_key: Optional[str] = None, output_dir: Optional[str] = None):
28
+ """Initialize the enhanced Gemini image generator.
29
 
30
  Args:
31
  api_key: Google Gemini API key
 
43
  self.output_dir = output_dir or os.path.join(os.path.expanduser("~"), "patent_architect_figures")
44
  os.makedirs(self.output_dir, exist_ok=True)
45
 
46
+ # Check if LaTeX is available for compilation
47
+ self.latex_available = self._check_latex_installation()
48
+
49
  # Image generation parameters
50
  self.generation_config = {
51
  'temperature': 0.4,
 
54
  'max_output_tokens': 8192,
55
  }
56
 
57
+ def _check_latex_installation(self) -> bool:
58
+ """Check if LaTeX is available for compilation."""
59
+ try:
60
+ result = subprocess.run(['pdflatex', '--version'],
61
+ capture_output=True,
62
+ text=True,
63
+ timeout=5)
64
+ return result.returncode == 0
65
+ except (subprocess.TimeoutExpired, FileNotFoundError, subprocess.SubprocessError):
66
+ return False
67
+
68
+ def _compile_latex_via_web_api(self, latex_code: str, filename: str) -> Optional[str]:
69
+ """Try to compile LaTeX using web-based APIs.
70
 
71
  Args:
72
+ latex_code: LaTeX source code
73
+ filename: Base filename for output
 
74
 
75
  Returns:
76
+ Path to compiled image or None if failed
77
  """
78
  try:
79
+ # Try using a public LaTeX compilation API (example with LaTeX.Online)
80
+ api_url = "https://latex.vercel.app/api/v2"
 
 
 
 
81
 
82
+ payload = {
83
+ "code": latex_code,
84
+ "format": "png",
85
+ "quality": 300
86
+ }
87
 
88
+ print(f"🌐 Attempting web-based LaTeX compilation...")
89
+ response = requests.post(api_url, json=payload, timeout=30)
90
+
91
+ if response.status_code == 200:
92
+ # Save compiled image
93
+ image_filename = f"{filename}.png"
94
+ image_path = os.path.join(self.output_dir, image_filename)
95
 
96
+ with open(image_path, 'wb') as f:
97
+ f.write(response.content)
 
 
 
 
 
98
 
99
+ print(f"✅ Web compilation successful: {image_filename}")
100
+ return image_path
101
+ else:
102
+ print(f"⚠️ Web compilation failed: HTTP {response.status_code}")
103
+ return None
104
 
105
+ except Exception as e:
106
+ print(f"⚠️ Web compilation error: {e}")
107
+ return None
108
+
109
+ def _compile_latex_to_image(self, latex_filepath: str) -> Optional[str]:
110
+ """Enhanced LaTeX compilation with multiple methods.
111
+
112
+ Args:
113
+ latex_filepath: Path to the .tex file
114
+
115
+ Returns:
116
+ Path to generated image file or None if compilation failed
117
+ """
118
+ # Try local compilation first if available
119
+ if self.latex_available:
120
+ try:
121
+ # Get directory and filename
122
+ tex_dir = os.path.dirname(latex_filepath)
123
+ tex_filename = os.path.basename(latex_filepath)
124
+ base_name = os.path.splitext(tex_filename)[0]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
125
 
126
+ print(f"🔧 Attempting local LaTeX compilation...")
127
+
128
+ # Compile LaTeX to PDF
129
+ result = subprocess.run([
130
+ 'pdflatex',
131
+ '-output-directory', tex_dir,
132
+ '-interaction=nonstopmode',
133
+ latex_filepath
134
+ ], capture_output=True, text=True, timeout=30)
135
+
136
+ if result.returncode == 0:
137
+ # Check if PDF was created
138
+ pdf_path = os.path.join(tex_dir, f"{base_name}.pdf")
139
+ if os.path.exists(pdf_path):
140
+ # Try to convert PDF to PNG using available tools
141
+ png_path = os.path.join(tex_dir, f"{base_name}.png")
142
 
143
+ # Try different conversion methods
144
+ conversion_methods = [
145
+ # Method 1: pdftoppm (part of poppler-utils)
146
+ ['pdftoppm', '-png', '-singlefile', '-r', '300', pdf_path, base_name],
147
+ # Method 2: ImageMagick convert
148
+ ['convert', '-density', '300', pdf_path, png_path],
149
+ # Method 3: ghostscript
150
+ ['gs', '-dNOPAUSE', '-dBATCH', '-sDEVICE=png16m', '-r300', f'-sOutputFile={png_path}', pdf_path]
151
+ ]
152
 
153
+ for method in conversion_methods:
154
+ try:
155
+ if method[0] == 'pdftoppm':
156
+ result = subprocess.run(method,
157
+ cwd=tex_dir,
158
+ capture_output=True,
159
+ text=True,
160
+ timeout=30)
161
+ if result.returncode == 0:
162
+ expected_png = os.path.join(tex_dir, f"{base_name}.png")
163
+ if os.path.exists(expected_png):
164
+ print(f"✅ Local compilation successful: {expected_png}")
165
+ return expected_png
166
+ else:
167
+ result = subprocess.run(method,
168
+ capture_output=True,
169
+ text=True,
170
+ timeout=30)
171
+ if result.returncode == 0 and os.path.exists(png_path):
172
+ print(f"✅ Local compilation successful: {png_path}")
173
+ return png_path
174
+
175
+ except (subprocess.TimeoutExpired, FileNotFoundError):
176
+ continue
177
 
178
+ print(f"✅ PDF created but PNG conversion failed: {pdf_path}")
179
+ return pdf_path # Return PDF if PNG conversion failed
180
+ else:
181
+ print(f"⚠️ Local LaTeX compilation failed: {result.stderr}")
182
+
183
+ except Exception as e:
184
+ print(f"⚠️ Local LaTeX compilation error: {e}")
185
+
186
+ # Try web-based compilation as fallback
187
+ try:
188
+ with open(latex_filepath, 'r', encoding='utf-8') as f:
189
+ latex_code = f.read()
 
 
 
 
 
 
 
 
 
 
 
 
190
 
191
+ base_name = os.path.splitext(os.path.basename(latex_filepath))[0]
192
+ web_result = self._compile_latex_via_web_api(latex_code, base_name)
193
+ if web_result:
194
+ return web_result
195
+ except Exception as e:
196
+ print(f"⚠️ Web compilation fallback failed: {e}")
197
+
198
+ return None
199
+
200
+ def generate_dual_output_figure(self,
201
+ invention_description: str,
202
+ figure_type: str = "technical_diagram",
203
+ specific_requirements: Optional[str] = None) -> Dict:
204
+ """Generate both detailed description AND LaTeX code for a patent figure.
205
+
206
+ Args:
207
+ invention_description: Description of the invention
208
+ figure_type: Type of figure to generate
209
+ specific_requirements: Additional requirements for the figure
210
+
211
+ Returns:
212
+ Dict containing both text description and LaTeX code with compilation options
213
+ """
214
+ try:
215
+ timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
216
+
217
+ # Step 1: Generate detailed text description
218
+ print(f"📝 Generating detailed technical description...")
219
  text_prompt = self._create_patent_figure_prompt(
220
  invention_description,
221
  figure_type,
222
  specific_requirements
223
  )
224
 
225
+ text_response = self.model.generate_content([text_prompt])
226
+ detailed_description = text_response.text
227
+
228
+ # Step 2: Generate LaTeX/TikZ code
229
+ print(f"🎨 Generating LaTeX/TikZ code...")
230
+ latex_prompt = self._create_image_generation_prompt(
231
+ invention_description,
232
+ figure_type,
233
+ specific_requirements
234
+ )
235
+
236
+ latex_response = self.model.generate_content([latex_prompt],
237
+ generation_config={'temperature': 0.1})
238
+ latex_code = latex_response.text
239
 
240
+ # Save both outputs
241
+ text_filename = f"patent_figure_{figure_type}_{timestamp}_description.txt"
242
+ latex_filename = f"patent_figure_{figure_type}_{timestamp}.tex"
243
 
244
+ text_filepath = os.path.join(self.output_dir, text_filename)
245
+ latex_filepath = os.path.join(self.output_dir, latex_filename)
246
+
247
+ # Save detailed description
248
+ with open(text_filepath, 'w', encoding='utf-8') as f:
249
  f.write(f"Patent Figure Description\n")
250
  f.write(f"========================\n\n")
251
  f.write(f"Invention: {invention_description}\n\n")
252
  f.write(f"Figure Type: {figure_type}\n\n")
253
+ f.write(f"Description:\n{detailed_description}\n\n")
254
  f.write(f"Generated: {datetime.now().isoformat()}\n")
255
 
256
+ # Save LaTeX code
257
+ with open(latex_filepath, 'w', encoding='utf-8') as f:
258
+ f.write(f"% Patent Figure - {figure_type}\n")
259
+ f.write(f"% Generated: {datetime.now().isoformat()}\n")
260
+ f.write(f"% Invention: {invention_description}\n\n")
261
+ f.write(latex_code)
262
+
263
+ # Try to compile LaTeX to image
264
+ compiled_image_path = None
265
+ compilation_method = None
266
+
267
+ if any(keyword in latex_code.lower() for keyword in ['tikz', 'latex', 'documentclass', '\\draw']):
268
+ compiled_image_path = self._compile_latex_to_image(latex_filepath)
269
+ if compiled_image_path:
270
+ if self.latex_available:
271
+ compilation_method = "local"
272
+ else:
273
+ compilation_method = "web"
274
 
275
  return {
276
  "success": True,
277
+ "dual_output": True,
278
+ "text_description": detailed_description,
279
+ "latex_code": latex_code,
280
+ "text_file_path": text_filepath,
281
+ "latex_file_path": latex_filepath,
282
+ "compiled_image_path": compiled_image_path,
283
+ "compilation_method": compilation_method,
284
  "figure_type": figure_type,
285
  "timestamp": timestamp,
286
+ "code_preview": latex_code[:400] + "..." if len(latex_code) > 400 else latex_code,
287
+ "description_preview": detailed_description[:300] + "..." if len(detailed_description) > 300 else detailed_description,
288
  "metadata": {
289
  "invention": invention_description,
290
  "requirements": specific_requirements,
291
+ "latex_available": self.latex_available,
292
+ "output_formats": ["description", "latex", "compiled_image"] if compiled_image_path else ["description", "latex"]
293
  }
294
  }
295
 
 
300
  "figure_type": figure_type
301
  }
302
 
303
+ def generate_patent_figure(self,
304
+ invention_description: str,
305
+ figure_type: str = "technical_diagram",
306
+ specific_requirements: Optional[str] = None) -> Dict:
307
+ """Enhanced patent figure generation with dual output option."""
308
+
309
+ # Use the new dual output method for better results
310
+ return self.generate_dual_output_figure(invention_description, figure_type, specific_requirements)
311
+
312
  def generate_multiple_figures(self,
313
  invention_description: str,
314
  figure_types: List[str] = None) -> List[Dict]:
 
402
  invention_description: str,
403
  figure_type: str,
404
  specific_requirements: Optional[str] = None) -> str:
405
+ """Create a prompt specifically for LaTeX/TikZ code generation.
406
 
407
  Args:
408
  invention_description: Description of the invention
 
410
  specific_requirements: Additional requirements
411
 
412
  Returns:
413
+ Optimized prompt for LaTeX/TikZ patent figure generation
414
  """
415
+ base_prompt = f"""Generate ONLY LaTeX/TikZ code for a professional patent figure showing: {invention_description}
416
 
417
  Figure Type: {figure_type}
418
 
419
+ OUTPUT REQUIREMENTS:
420
+ - Complete LaTeX document ready to compile
421
+ - Start with \\documentclass{{article}} and end with \\end{{document}}
422
+ - Include necessary packages: \\usepackage{{tikz}}, \\usepackage{{geometry}}
423
+ - Professional patent-style technical drawing
424
  - Numbered components (1, 2, 3, etc.) with clear labels
425
+ - Clean, precise TikZ drawing commands
426
+ - No explanatory text - ONLY LaTeX code
 
 
 
427
 
428
+ TECHNICAL DRAWING REQUIREMENTS:
429
+ - Use TikZ \\draw commands for all shapes
430
+ - Include numbered component labels with \\node commands
431
+ - Proper scaling and positioning
432
+ - Professional technical drawing style
433
+ - Clear spatial relationships between components
434
+ - Include connecting lines and annotations where relevant
435
 
436
  """
437
 
 
462
  def integrate_with_patent_architect(self,
463
  invention_description: str,
464
  figure_descriptions: List[str] = None) -> Dict:
465
+ """Enhanced integration point for Patent Architect AI v2 system with dual output.
466
 
467
  Args:
468
  invention_description: The invention being patented
469
  figure_descriptions: Specific figure descriptions needed
470
 
471
  Returns:
472
+ Dict with enhanced content including both descriptions and LaTeX code
473
  """
474
  try:
475
  if figure_descriptions is None:
 
478
  results = []
479
 
480
  for i, figure_type in enumerate(figure_types, 1):
481
+ result = self.generate_dual_output_figure(invention_description, figure_type)
482
  if result["success"]:
483
  results.append({
484
  "figure_number": i,
485
+ "description": result["text_description"],
486
+ "latex_code": result["latex_code"],
487
+ "text_file_path": result["text_file_path"],
488
+ "latex_file_path": result["latex_file_path"],
489
+ "compiled_image_path": result.get("compiled_image_path"),
490
+ "compilation_method": result.get("compilation_method"),
491
  "type": figure_type,
492
+ "dual_output": True,
493
+ "code_preview": result.get("code_preview", ""),
494
+ "description_preview": result.get("description_preview", "")
495
  })
496
  else:
497
  # Generate specific requested figures
498
  results = []
499
  for i, description in enumerate(figure_descriptions, 1):
500
+ result = self.generate_dual_output_figure(
501
  invention_description,
502
  f"custom_figure_{i}",
503
  description
 
505
  if result["success"]:
506
  results.append({
507
  "figure_number": i,
508
+ "description": result["text_description"],
509
+ "latex_code": result["latex_code"],
510
+ "text_file_path": result["text_file_path"],
511
+ "latex_file_path": result["latex_file_path"],
512
+ "compiled_image_path": result.get("compiled_image_path"),
513
+ "compilation_method": result.get("compilation_method"),
514
  "type": "custom",
515
+ "dual_output": True,
516
+ "code_preview": result.get("code_preview", ""),
517
+ "description_preview": result.get("description_preview", "")
518
  })
519
 
520
+ # Enhanced formatting for Patent Architect with dual output
521
  if results:
522
+ content = "## 🎨 Enhanced Patent Figures with Dual Output\n\n"
523
 
524
+ # Count outputs
 
 
525
  total_figures = len(results)
526
+ compiled_images = sum(1 for r in results if r.get("compiled_image_path"))
527
+ local_compiled = sum(1 for r in results if r.get("compilation_method") == "local")
528
+ web_compiled = sum(1 for r in results if r.get("compilation_method") == "web")
529
 
530
+ content += f"*✨ Generated {total_figures} complete figure sets with both detailed descriptions AND LaTeX/TikZ code*\n\n"
531
+
532
+ if compiled_images > 0:
533
+ content += f"*🖼️ {compiled_images}/{total_figures} figures successfully compiled to images"
534
+ if local_compiled > 0:
535
+ content += f" ({local_compiled} local"
536
+ if web_compiled > 0:
537
+ if local_compiled > 0:
538
+ content += f", {web_compiled} web"
539
+ else:
540
+ content += f" ({web_compiled} web"
541
+ content += " compilation)*\n\n"
542
 
543
  for figure in results:
544
  figure_title = figure['type'].replace('_', ' ').title()
545
  content += f"### Figure {figure['figure_number']}: {figure_title}\n\n"
546
 
547
+ # Show compiled image if available
548
+ if figure.get("compiled_image_path"):
549
+ image_filename = os.path.basename(figure['compiled_image_path'])
550
+ content += f"**🖼️ Compiled Image:** `{image_filename}` ({'Local' if figure.get('compilation_method') == 'local' else 'Web'} compilation)\n\n"
551
+
552
+ # Always show detailed description
553
+ content += f"**📝 Technical Description:**\n"
554
+ content += f"{figure['description'][:500]}{'...' if len(figure['description']) > 500 else ''}\n\n"
555
+
556
+ # Always show LaTeX code with enhanced instructions
557
+ latex_filename = os.path.basename(figure['latex_file_path'])
558
+ content += f"**🎨 LaTeX/TikZ Code:** `{latex_filename}`\n\n"
559
+
560
+ # Show code preview
561
+ if figure.get('code_preview'):
562
+ content += f"**Code Preview:**\n```latex\n{figure['code_preview']}\n```\n\n"
563
+
564
+ # Enhanced compilation options
565
+ content += f"**🚀 Multiple Viewing Options:**\n\n"
566
+
567
+ if figure.get("compiled_image_path"):
568
+ content += f"1. **✅ Pre-Compiled Image Available**: View `{os.path.basename(figure['compiled_image_path'])}` in output folder\n"
569
+
570
+ content += f"2. **🌐 Online Compilation (Easiest)**:\n"
571
+ content += f" - Go to [Overleaf.com](https://overleaf.com) (free account)\n"
572
+ content += f" - Create new project → Upload → Select `{latex_filename}`\n"
573
+ content += f" - Click 'Recompile' for instant professional PDF\n"
574
+ content += f" - Download PDF for USPTO submission\n\n"
575
+
576
+ content += f"3. **💻 Local Compilation**:\n"
577
+ content += f" - Install LaTeX: [MiKTeX](https://miktex.org/) (Windows) or [TeX Live](https://www.tug.org/texlive/)\n"
578
+ content += f" - Run: `pdflatex {latex_filename}`\n"
579
+ content += f" - Convert to PNG: `pdftoppm -png -singlefile output.pdf figure`\n\n"
580
+
581
+ content += f"4. **📱 Mobile/Quick View**: Use [LaTeX Live](https://latexlive.com) - paste code for instant preview\n\n"
582
+
583
+ # File information
584
+ content += f"**📁 Generated Files:**\n"
585
+ content += f"- 📝 Description: `{os.path.basename(figure['text_file_path'])}`\n"
586
+ content += f"- 🎨 LaTeX Code: `{latex_filename}`\n"
587
+ if figure.get("compiled_image_path"):
588
+ content += f"- 🖼️ Compiled Image: `{os.path.basename(figure['compiled_image_path'])}`\n"
589
+ content += f"- 📂 Location: `{os.path.dirname(figure['text_file_path'])}`\n\n"
590
 
591
+ # Enhanced summary and guidance
592
  content += "---\n\n"
593
+ content += f"**📊 Generation Summary:**\n"
594
+ content += f"- **Total Figures:** {total_figures} complete sets\n"
595
+ content += f"- **Dual Output:** ✅ Both descriptions AND LaTeX code\n"
596
+ content += f"- **Compiled Images:** {compiled_images} ready to view\n"
597
+ content += f"- **USPTO Ready:** ✅ Professional patent figure quality\n"
598
+ content += f"- **Vector Graphics:** Infinite scaling, publication quality\n\n"
599
 
600
+ content += f"**🏆 Why This is Better:**\n"
601
+ content += f"- **Immediate Understanding:** Read detailed technical descriptions right now\n"
602
+ content += f"- **Professional Quality:** LaTeX creates publication-grade vector graphics\n"
603
+ content += f"- **Multiple Options:** View online (Overleaf), locally, or pre-compiled images\n"
604
+ content += f"- **USPTO Preferred:** Vector graphics are ideal for patent applications\n"
605
+ content += f"- **Editable:** Modify LaTeX code for custom adjustments\n\n"
606
+
607
+ content += f"**🎯 Quick Start Guide:**\n"
608
+ content += f"1. **Read the descriptions above** for immediate technical understanding\n"
609
+ content += f"2. **View compiled images** if available in the output folder\n"
610
+ content += f"3. **For publication quality**: Use Overleaf.com with the .tex files\n"
611
+ content += f"4. **For USPTO submission**: Compile LaTeX to PDF for vector graphics\n\n"
612
 
613
+ content += f"*Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')} | Output: `{self.output_dir}`*\n\n"
614
 
615
  return {
616
  "success": True,
617
  "content": content,
618
  "figures": results,
619
  "total_figures": len(results),
620
+ "dual_output": True,
621
+ "compiled_images": compiled_images,
622
+ "descriptions_available": total_figures,
623
+ "latex_code_available": total_figures,
624
+ "enhanced_features": [
625
+ "detailed_descriptions",
626
+ "latex_tikz_code",
627
+ "multiple_compilation_options",
628
+ "pre_compiled_images",
629
+ "overleaf_integration",
630
+ "uspto_ready_output"
631
+ ]
632
  }
633
  else:
634
  return {
635
  "success": False,
636
+ "content": "### Technical Figures\n\nError generating enhanced patent figures. Please review invention description and try again.\n\n",
637
  "error": "No figures were successfully generated"
638
  }
639
 
 
677
  print(f" 🎨 Format: {result.get('mime_type', 'Unknown')}")
678
  else:
679
  print(f"ℹ️ Generated detailed text description (image generation unavailable)")
680
+ print(f" 📄 Description file: {result['text_file_path']}")
681
  else:
682
  print(f"❌ Individual generation failed: {result.get('error', 'Unknown error')}")
683
 
 
762
  print(f" 📎 Has inline data: ✅")
763
  print(f" 🎨 MIME type: {part.inline_data.mime_type}")
764
  else:
765
+ print(f" �� Has inline data: ❌")
766
 
767
  if hasattr(response, 'text') and response.text:
768
  preview = response.text[:200] + "..." if len(response.text) > 200 else response.text