<p>Embedding is necessary to convert text in vectors that are more tractable by machine learning models.</p>
<p>Can embedding be carried out (efficiently) using the MapReduce approach?</p>
<p>I suspect so: projecting text in another space shouldn't be much harder than relational algebra operations such as joins, which can be done efficiently using MapReduce.</p>
<p>Is there any source on this?</p>

CC BY-SA 4.0
2024-06-05T14:02:44.710Z
it's literally just a map, but you need to share the dictionary across workers, which might be expensive
Alberto
CC BY-SA 4.0
2024-06-05T18:51:30.183Z
@Alberto I'm aware that MR is essentially Turing-complete, so anything can be done using MR. My question is whether or not a distributed MR-style embedding is advantageous.
A. Darwin
CC BY-SA 4.0
2024-06-05T21:15:41.840Z
I see, you are probably looking for this paper https://arxiv.org/pdf/1606.08495
Alberto
<embeddings>
Can embedding be tackled efficiently using MapReduce?
A. Darwin
# Can embedding be tackled efficiently using MapReduce?

(Asked by: username_0 on 2024-06-04)

Embedding is necessary to convert text in vectors that are more tractable by machine learning models.

Can embedding be carried out (efficiently) using the MapReduce approach?

I suspect so: projecting text in another space shouldn't be much harder than relational algebra operations such as joins, which can be done efficiently using MapReduce.

Is there any source on this?

**Tags:** embeddings

> **Comments:**
> **username_1**: it's literally just a map, but you need to share the dictionary across workers, which might be expensive  
> **username_0**: @username_1 I'm aware that MR is essentially Turing-complete, so anything can be done using MR. My question is whether or not a distributed MR-style embedding is advantageous.  
> **username_1**: I see, you are probably looking for this paper https://arxiv.org/pdf/1606.08495  

---

## 0 answers

