Qwen/Qwen-72B
Text Generation
•
Updated
•
2.72k
•
344
Marcelo Cardoso
marcelovicentegc
AI & ML interests
Natural language processing and code generation
Recent Activity
liked
a model
22 days ago
Qwen/Qwen-72B
liked
a model
23 days ago
Qwen/Qwen2-VL-72B-Instruct
liked
a model
24 days ago
deepseek-ai/DeepSeek-Coder-V2-Instruct
Organizations
marcelovicentegc's activity
liked
a
model
22 days ago
liked
a
model
23 days ago
liked
4
models
24 days ago
deepseek-ai/DeepSeek-Coder-V2-Instruct
Text Generation
•
Updated
•
105k
•
512
deepseek-ai/deepseek-coder-33b-instruct
Text Generation
•
Updated
•
3.77k
•
463
Qwen/Qwen2.5-Coder-32B-Instruct
Text Generation
•
Updated
•
375k
•
•
1.37k
WizardLMTeam/WizardCoder-15B-V1.0
Text Generation
•
Updated
•
1.51k
•
749
liked
a
dataset
24 days ago
reacted to
akhaliq's
post with ❤️
10 months ago
Post
Design2Code
How Far Are We From Automating Front-End Engineering?
Design2Code: How Far Are We From Automating Front-End Engineering? (2403.03163)
Generative AI has made rapid advancements in recent years, achieving unprecedented capabilities in multimodal understanding and code generation. This can enable a new paradigm of front-end development, in which multimodal LLMs might directly convert visual designs into code implementations. In this work, we formalize this as a Design2Code task and conduct comprehensive benchmarking. Specifically, we manually curate a benchmark of 484 diverse real-world webpages as test cases and develop a set of automatic evaluation metrics to assess how well current multimodal LLMs can generate the code implementations that directly render into the given reference webpages, given the screenshots as input. We also complement automatic metrics with comprehensive human evaluations. We develop a suite of multimodal prompting methods and show their effectiveness on GPT-4V and Gemini Pro Vision. We further finetune an open-source Design2Code-18B model that successfully matches the performance of Gemini Pro Vision. Both human evaluation and automatic metrics show that GPT-4V performs the best on this task compared to other models. Moreover, annotators think GPT-4V generated webpages can replace the original reference webpages in 49% of cases in terms of visual appearance and content; and perhaps surprisingly, in 64% of cases GPT-4V generated webpages are considered better than the original reference webpages. Our fine-grained break-down metrics indicate that open-source models mostly lag in recalling visual elements from the input webpages and in generating correct layout designs, while aspects like text content and coloring can be drastically improved with proper finetuning.
How Far Are We From Automating Front-End Engineering?
Design2Code: How Far Are We From Automating Front-End Engineering? (2403.03163)
Generative AI has made rapid advancements in recent years, achieving unprecedented capabilities in multimodal understanding and code generation. This can enable a new paradigm of front-end development, in which multimodal LLMs might directly convert visual designs into code implementations. In this work, we formalize this as a Design2Code task and conduct comprehensive benchmarking. Specifically, we manually curate a benchmark of 484 diverse real-world webpages as test cases and develop a set of automatic evaluation metrics to assess how well current multimodal LLMs can generate the code implementations that directly render into the given reference webpages, given the screenshots as input. We also complement automatic metrics with comprehensive human evaluations. We develop a suite of multimodal prompting methods and show their effectiveness on GPT-4V and Gemini Pro Vision. We further finetune an open-source Design2Code-18B model that successfully matches the performance of Gemini Pro Vision. Both human evaluation and automatic metrics show that GPT-4V performs the best on this task compared to other models. Moreover, annotators think GPT-4V generated webpages can replace the original reference webpages in 49% of cases in terms of visual appearance and content; and perhaps surprisingly, in 64% of cases GPT-4V generated webpages are considered better than the original reference webpages. Our fine-grained break-down metrics indicate that open-source models mostly lag in recalling visual elements from the input webpages and in generating correct layout designs, while aspects like text content and coloring can be drastically improved with proper finetuning.
- 4 replies
liked
2
models
10 months ago
liked
a
dataset
10 months ago
liked
4
models
10 months ago
liked
3
models
11 months ago
liked
2
models
about 1 year ago
