MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility
  Applications
Accessibility remains a critical concern in today's society, as many
technologies are not developed to support the full range of user needs.
Existing multi-agent systems (MAS) often cannot provide comprehensive
assistance for users in need due to the lack of customization stemming from
closed-source designs. Consequently, individuals with disabilities frequently
encounter significant barriers when attempting to interact with digital
environments. We introduce MATE, a multimodal accessibility MAS, which performs
the modality conversions based on the user's needs. The system is useful for
assisting people with disabilities by ensuring that data will be converted to
an understandable format. For instance, if the user cannot see well and
receives an image, the system converts this image to its audio description.
MATE can be applied to a wide range of domains, industries, and areas, such as
healthcare, and can become a useful assistant for various groups of users. The
system supports multiple types of models, ranging from LLM API calling to using
custom machine learning (ML) classifiers. This flexibility ensures that the
system can be adapted to various needs and is compatible with a wide variety of
hardware. Since the system is expected to run locally, it ensures the privacy
and security of sensitive information. In addition, the framework can be
effectively integrated with institutional technologies (e.g., digital
healthcare service) for real-time user assistance. Furthermore, we introduce
ModCon-Task-Identifier, a model that is capable of extracting the precise
modality conversion task from the user input. Numerous experiments show that
ModCon-Task-Identifier consistently outperforms other LLMs and statistical
models on our custom data. Our code and data are publicly available at
https://github.com/AlgazinovAleksandr/Multi-Agent-MATE.
685c02aadf8a0d6c70bbf918
Aleksandr Algazinov
claimed_verified
2025-06-25T20:59:09.301Z
674ed490fc7f50ef61c3a7bd
https://cdn-avatars.huggingface.co/v1/production/uploads/674ed490fc7f50ef61c3a7bd/j260ICQmNl42aXehBEd6P.jpeg
Aleksandr Algazinov
user
AleksandrAlgazinov
685c02aadf8a0d6c70bbf919
Matt Laing
685c02aadf8a0d6c70bbf91a
Paul Laban
MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility
  Applications
Accessibility remains a critical concern in today's society, as many
technologies are not developed to support the full range of user needs.
Existing multi-agent systems (MAS) often cannot provide comprehensive
assistance for users in need due to the lack of customization stemming from
closed-source designs. Consequently, individuals with disabilities frequently
encounter significant barriers when attempting to interact with digital
environments. We introduce MATE, a multimodal accessibility MAS, which performs
the modality conversions based on the user's needs. The system is useful for
assisting people with disabilities by ensuring that data will be converted to
an understandable format. For instance, if the user cannot see well and
receives an image, the system converts this image to its audio description.
MATE can be applied to a wide range of domains, industries, and areas, such as
healthcare, and can become a useful assistant for various groups of users. The
system supports multiple types of models, ranging from LLM API calling to using
custom machine learning (ML) classifiers. This flexibility ensures that the
system can be adapted to various needs and is compatible with a wide variety of
hardware. Since the system is expected to run locally, it ensures the privacy
and security of sensitive information. In addition, the framework can be
effectively integrated with institutional technologies (e.g., digital
healthcare service) for real-time user assistance. Furthermore, we introduce
ModCon-Task-Identifier, a model that is capable of extracting the precise
modality conversion task from the user input. Numerous experiments show that
ModCon-Task-Identifier consistently outperforms other LLMs and statistical
models on our custom data. Our code and data are publicly available at
https://github.com/AlgazinovAleksandr/Multi-Agent-MATE.
MATE, a multimodal accessibility multi-agent system, converts data into understandable formats based on user needs, supporting various disabilities and integrating with institutional technologies.
MULTIAGENT SYSTEMS MAS MODALITY CONVERSIONS LLM API CUSTOM MACHINE LEARNING CLASSIFIERS MODCON-TASK-IDENTIFIER LLM STATISTICAL MODELS