Abstract
Advances in transformer-based architectures have enabled Large Language Models to generate fluent, coherent, diverse, and consistent text, comparable to human-produced content. However, the main efforts in this area remain predominantly focused on English-language models, raising concerns about their inclusivity and applicability across diverse linguistic and domain-specific contexts. Thus, this study presents a comprehensive literature review aimed to address the research question: ”What types of techniques are applied in the field of natural language processing for text generation?”. Unlike earlier surveys, which typically concentrate on specific tasks or techniques, this review examines applied techniques, covered languages, data domains, and the extent to which proposed models qualify as foundation models. A total of 263 peer-reviewed studies published between 2017 and 2025 were identified and analysed. The review identified 20 types of techniques applied to automatic text generation across 26 languages and 27 application domains. Findings reveal a strong prevalence of general-purpose English-language models, whilst research on multilingual and domain-specific models remains limited. Only thirteen models were identified as foundation models, indicating a lack of broader generalisation capabilities. Key gaps include the absence of foundation models for non-English languages and insufficient exploration of specific domains such as healthcare, law, and finance. Additional challenges involve dataset availability, computational costs, and methodological transparency. Emerging trends suggest a growing focus on multilingual and domain-adapted models, transfer learning, and reinforcement learning with human feedback. Future opportunities lie in expanding research on unexplored languages and domains, addressing ethical concerns, improving evaluation metrics, and developing responsible, people-centred AI systems.
[Display omitted]
•A total of 313 papers was reviewed, identifying 20 text generation techniques.•They cover applications across 26 languages and 27 domains.•Few models were trained for non-English languages or specialised domains.•Key research gaps were identified, stressing the need for broader model coverage.