Removing Duplicates with Map In JavaScript

Updated on November 18, 2019 · 3 min read|

It is quite common to use Set to remove duplicate items from an array. This can be achieved by wrapping an existing array into a Set constructor and then transforming it back into an array:

javascript
const arr = [1, 2, 'a', 'b', 2, 'a', 3, 4];
const uniqueArr = [...new Set(arr)];
 
console.log(uniqueArr); // [1, 2, "a", "b", 3, 4]

javascript
const arr = [1, 2, 'a', 'b', 2, 'a', 3, 4];
const uniqueArr = [...new Set(arr)];
 
console.log(uniqueArr); // [1, 2, "a", "b", 3, 4]

This works great for arrays of primitive values, however, when applying the same approach to an array of arrays or objects, the result is quite disappointing:

javascript
const arr = [[1, 2], {'a': 'b'}, {'a':2}, {'a':'b'}, [3, 4], [1, 2]];
const uniqueArr = [...new Set(arr)];
 
console.log(uniqueArr); // [[1, 2], {'a': 'b'}, {'a':2}, {'a':'b'}, [3, 4], [1, 2]]

javascript
const arr = [[1, 2], {'a': 'b'}, {'a':2}, {'a':'b'}, [3, 4], [1, 2]];
const uniqueArr = [...new Set(arr)];
 
console.log(uniqueArr); // [[1, 2], {'a': 'b'}, {'a':2}, {'a':'b'}, [3, 4], [1, 2]]

This is because Set compares non-primitive values by reference and not by value, and in our case, all values in the array have a different reference.

A bit less-known fact is that the Map data structure maintains key uniqueness, meaning that there can be no more than one key-value pair with the same key. While knowing this won't help us magically transform any array into an array of unique values, certain use cases can benefit from Map's key uniqueness.

Let's consider a sample React app, which displays a list of books and a dropdown that allows filtering books by their authors.

jsx
const App = () => {
  const books = [
    {
      id: 1,
      name: "In Search of Lost Time ",
      author: { name: "Marcel Proust", id: 1 }
    },
    { id: 2, name: "Ulysses", author: { name: "James Joyce", id: 2 } },
    {
      id: 3,
      name: "Don Quixote",
      author: { name: "Miguel de Cervantes", id: 3 }
    },
    { id: 4, name: "Hamlet", author: { name: "William Shakespeare", id: 4 } },
    {
      id: 5,
      name: "Romeo and Juliet",
      author: { name: "William Shakespeare", id: 4 }
    },
    { id: 6, name: "Dubliners", author: { name: "James Joyce", id: 2 } }
  ];
  const [selectedAuthorId, setSelectedAuthorId] = useState(null);
 
  const filteredBooks = () => {
    if (!selectedAuthorId) {
      return books;
    }
    return books.filter(book => String(book.author.id) === selectedAuthorId);
  };
 
  return (
    <div className="books">
      <select
        className="books__select"
        onChange={({ target }) => setSelectedAuthorId(target.value)}
      >
        {/*Show author options*/}
      </select>
      <ul className="books__list">
        {filteredBooks().map(book => (
          <li className="books__item">
            {book.name} by {book.author.name}
          </li>
        ))}
      </ul>
    </div>
  );
};

jsx
const App = () => {
  const books = [
    {
      id: 1,
      name: "In Search of Lost Time ",
      author: { name: "Marcel Proust", id: 1 }
    },
    { id: 2, name: "Ulysses", author: { name: "James Joyce", id: 2 } },
    {
      id: 3,
      name: "Don Quixote",
      author: { name: "Miguel de Cervantes", id: 3 }
    },
    { id: 4, name: "Hamlet", author: { name: "William Shakespeare", id: 4 } },
    {
      id: 5,
      name: "Romeo and Juliet",
      author: { name: "William Shakespeare", id: 4 }
    },
    { id: 6, name: "Dubliners", author: { name: "James Joyce", id: 2 } }
  ];
  const [selectedAuthorId, setSelectedAuthorId] = useState(null);
 
  const filteredBooks = () => {
    if (!selectedAuthorId) {
      return books;
    }
    return books.filter(book => String(book.author.id) === selectedAuthorId);
  };
 
  return (
    <div className="books">
      <select
        className="books__select"
        onChange={({ target }) => setSelectedAuthorId(target.value)}
      >
        {/*Show author options*/}
      </select>
      <ul className="books__list">
        {filteredBooks().map(book => (
          <li className="books__item">
            {book.name} by {book.author.name}
          </li>
        ))}
      </ul>
    </div>
  );
};

For simplicity books array is hardcoded here, although in a real-world app, the data will be probably fetched from an API.

The app is almost complete, we just need to render the dropdown of authors to filter by. A good way to approach it would be to collect the id and name of each author from our list of books into a separate array and render it as options inside the select. However, there's one condition - this list should contain unique authors, otherwise authors of more than one book will appear in the dropdown multiple times, which is something we don't want to happen. We need both id for the option's value and name to display the option's label and since the author's data is contained inside an object we can't just apply the Set trick to get unique values only. One option would be to first get all the ids for the authors into an array, then apply Set to it to get unique ones, and after that iterate over the authors array once more to collect their names based on the ids. That sounds like a lot of work, and luckily there's an easier solution.

Considering that we need an array of id - name pairs, we can extract those from the books list and transform them into a Map, which would automatically take care of preserving only the pairs with unique keys.

javascript
  const authorOptions = new Map([
    ...books.map(book => [book.author.id, book.author.name])
  ]);

javascript
  const authorOptions = new Map([
    ...books.map(book => [book.author.id, book.author.name])
  ]);

That's it! Now we have a Map of unique key-value pairs, which we can feed directly into our select component.

It's worth keeping in mind that when Map preserves key uniqueness, the last inserted item with the existing key stays in the Map, while previous duplicates are discarded.

javascript
const map1 = new Map([[1,3], [2,3]]);
 
const map2 = new Map([[1,2]]);
 
var merged = new Map([...map1, ...map2]);
 
console.log(merged.get(1)); // 2
console.log(merged.get(2)); // 3

javascript
const map1 = new Map([[1,3], [2,3]]);
 
const map2 = new Map([[1,2]]);
 
var merged = new Map([...map1, ...map2]);
 
console.log(merged.get(1)); // 2
console.log(merged.get(2)); // 3

Luckily, in our example app all the author id - name pairs are unique so we don't need to worry about accidentally overriding any data.

Now we can combine everything into the final version of the component.

jsx
const App = () => {
  const books = [
    {
      id: 1,
      name: "In Search of Lost Time ",
      author: { name: "Marcel Proust", id: 1 }
    },
    { id: 2, name: "Ulysses", author: { name: "James Joyce", id: 2 } },
    {
      id: 3,
      name: "Don Quixote",
      author: { name: "Miguel de Cervantes", id: 3 }
    },
    { id: 4, name: "Hamlet", author: { name: "William Shakespeare", id: 4 } },
    {
      id: 5,
      name: "Romeo and Juliet",
      author: { name: "William Shakespeare", id: 4 }
    },
    { id: 6, name: "Dubliners", author: { name: "James Joyce", id: 2 } }
  ];
  const [selectedAuthorId, setSelectedAuthorId] = useState(null);
 
  const authorOptions = new Map([
    ...books.map(book => [book.author.id, book.author.name])
  ]);
 
  const filteredBooks = () => {
    if (!selectedAuthorId) {
      return books;
    }
    return books.filter(book => String(book.author.id) === selectedAuthorId);
  };
 
  return (
    <div className="books">
      <select
        className="books__select"
        onChange={({ target }) => setSelectedAuthorId(target.value)}
      >
         <option value=''>--Select author--</option>
        {[...authorOptions].map(([id, name]) => (
          <option value={id}>{name}</option>
        ))}
      </select>
      <ul className="books__list">
        {filteredBooks().map(book => (
          <li className="books__item">
            {book.name} by {book.author.name}
          </li>
        ))}
      </ul>
    </div>
  );
};

jsx
const App = () => {
  const books = [
    {
      id: 1,
      name: "In Search of Lost Time ",
      author: { name: "Marcel Proust", id: 1 }
    },
    { id: 2, name: "Ulysses", author: { name: "James Joyce", id: 2 } },
    {
      id: 3,
      name: "Don Quixote",
      author: { name: "Miguel de Cervantes", id: 3 }
    },
    { id: 4, name: "Hamlet", author: { name: "William Shakespeare", id: 4 } },
    {
      id: 5,
      name: "Romeo and Juliet",
      author: { name: "William Shakespeare", id: 4 }
    },
    { id: 6, name: "Dubliners", author: { name: "James Joyce", id: 2 } }
  ];
  const [selectedAuthorId, setSelectedAuthorId] = useState(null);
 
  const authorOptions = new Map([
    ...books.map(book => [book.author.id, book.author.name])
  ]);
 
  const filteredBooks = () => {
    if (!selectedAuthorId) {
      return books;
    }
    return books.filter(book => String(book.author.id) === selectedAuthorId);
  };
 
  return (
    <div className="books">
      <select
        className="books__select"
        onChange={({ target }) => setSelectedAuthorId(target.value)}
      >
         <option value=''>--Select author--</option>
        {[...authorOptions].map(([id, name]) => (
          <option value={id}>{name}</option>
        ))}
      </select>
      <ul className="books__list">
        {filteredBooks().map(book => (
          <li className="books__item">
            {book.name} by {book.author.name}
          </li>
        ))}
      </ul>
    </div>
  );
};

P.S.: Are you looking for a reliable and user-friendly hosting solution for your website or blog? Cloudways is a managed cloud platform that offers hassle-free and high-performance hosting experience. With 24/7 support and a range of features, Cloudways is a great choice for any hosting needs.

Removing Duplicates with Map In JavaScript

Related posts