Industry 4.0 encourages industries to digitise the manufacturing system to facilitate human-robot collaboration (HRC) to foster efficiency, agility and resilience. This cutting-edge technology strikes a balance between fully automated and manual operations to maximise the benefits of both humans and assistant robots (known as cobots) working together on complicated and prone-to-hazardous tasks in a collaborative manner in an assembly system. However, the introduction of HRC poses a significant challenge for assembly line balancing since, besides typical assigning tasks to workstations, the other two important decisions must also be made regarding equipping workstations with appropriate cobots as well as scheduling collaborative tasks for workers and cobots. In this article, the cobot assembly line balancing problem (CoALBP), which just initially emerged a few years ago, is thoroughly reviewed. The 4M1E (i.e., man, machine, material, method and environment) framework is applied for categorising the problem to make the review process more effective. All of the articles reviewed are compared, and their key distinct features are summarised. Finally, guidelines for additional studies on the CoALBP are offered.