Repository logo
 
Publication

On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms

2023Journal article Open access
Simple item page
dc.contributor.authorMoura, André
dc.contributor.authorLima, Pedro
dc.contributor.authorMendonça, Fábio
dc.contributor.authorMostafa, Sheikh Shanawaz
dc.contributor.authorDias, Fernando Morgado
dc.date.accessioned2024-02-16T11:41:06Z
dc.date.available2024-02-16T11:41:06Z
dc.date.issued2023
dc.description.abstractChatbots are becoming increasingly popular and require the ability to interpret natural language to provide clear communication with humans. To achieve this, intent detection is cru cial. However, current applications typically need a significant amount of annotated data, which is time-consuming and expensive to acquire. This article assesses the effectiveness of different text representations for annotating unlabeled dialog data through a pipeline that examines both classical approaches and pre-trained transformer models for word embedding. The resulting embeddings were then used to create sentence embeddings through pooling, followed by dimensionality re duction, before being fed into a clustering algorithm to determine the user’s intents. Therefore, various pooling, dimension reduction, and clustering algorithms were evaluated to determine the most appropriate approach. The evaluation dataset contains a variety of user intents across differ ent domains, with varying intent taxonomies within the same domain. Results demonstrate that transformer-based models perform better text representation than classical approaches. However, combining several clustering algorithms and embeddings from dissimilar origins through ensemble clustering considerably improves the final clustering solution. Additionally, applying the uniform manifold approximation and projection algorithm for dimension reduction can substantially improve performance (up to 20%) while using a much smaller representation.pt_PT
dc.description.versioninfo:eu-repo/semantics/publishedVersionpt_PT
dc.identifier.citationMoura, A.; Lima, P.; Mendonça, F.; Mostafa, S.S.; Dias, F. M. On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms. Appl. Sci. 2023, 13, 5178. https://doi.org/10.3390/app13085178pt_PT
dc.identifier.doi10.3390/app13085178pt_PT
dc.identifier.urihttp://hdl.handle.net/10400.13/5558
dc.language.isoengpt_PT
dc.peerreviewedyespt_PT
dc.publisherMDPIpt_PT
dc.relationLaboratory of Robotics and Engineering Systems
dc.relationLaboratory for Robotics and Engineering Systems
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/pt_PT
dc.subjectBERTpt_PT
dc.subjectChatbotspt_PT
dc.subjectEmbedding clusteringpt_PT
dc.subjectIntent detectionpt_PT
dc.subjectNatural language processingpt_PT
dc.subjectNatural language understandingpt_PT
dc.subjectRoBERTapt_PT
dc.subjectWord and sentence embeddingpt_PT
dc.subject.pt_PT
dc.subjectFaculdade de Ciências Exatas e da Engenhariapt_PT
dc.titleOn the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithmspt_PT
dc.typejournal article
dspace.entity.typePublication
oaire.awardTitleLaboratory of Robotics and Engineering Systems
oaire.awardTitleLaboratory for Robotics and Engineering Systems
oaire.awardURIinfo:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F50009%2F2020/PT
oaire.awardURIinfo:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UID%2FEEA%2F50009%2F2019/PT
oaire.citation.issue8pt_PT
oaire.citation.startPage5178pt_PT
oaire.citation.titleApplied Sciencespt_PT
oaire.citation.volume13pt_PT
oaire.fundingStream6817 - DCRRNI ID
oaire.fundingStream6817 - DCRRNI ID
person.familyNameSilva Mendonça
person.familyNameMostafa
person.familyNameMorgado-Dias
person.givenNameFábio Rúben
person.givenNameSheikh Shanawaz
person.givenNameFernando
person.identifier34497
person.identifier.ciencia-id7F1E-8AE9-3098
person.identifier.ciencia-idEE14-BEB3-F82B
person.identifier.ciencia-id7B14-DF07-AA6D
person.identifier.orcid0000-0002-5107-3248
person.identifier.orcid0000-0002-7677-0971
person.identifier.orcid0000-0001-7334-3993
person.identifier.ridN-9228-2015
person.identifier.scopus-author-id55489640900
project.funder.identifierhttp://doi.org/10.13039/501100001871
project.funder.identifierhttp://doi.org/10.13039/501100001871
project.funder.nameFundação para a Ciência e a Tecnologia
project.funder.nameFundação para a Ciência e a Tecnologia
rcaap.rightsopenAccesspt_PT
rcaap.typearticlept_PT
relation.isAuthorOfPublication0dbad4bf-3a0c-4c43-8022-42199a5e09c0
relation.isAuthorOfPublicationf90aafd0-eedb-47ea-945a-40b1c1fe802a
relation.isAuthorOfPublication042f7593-c6ca-4553-8f0e-12ccf17018be
relation.isAuthorOfPublication.latestForDiscovery0dbad4bf-3a0c-4c43-8022-42199a5e09c0
relation.isProjectOfPublicationc0352e81-99e6-4923-99ed-074df09e4db0
relation.isProjectOfPublicationeda3c844-853e-4c97-83da-99235180d98c
relation.isProjectOfPublication.latestForDiscoveryeda3c844-853e-4c97-83da-99235180d98c

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
On the Use of Transformer-Based Models for Intent Detection Using Clustering Algorithms.pdf
Size:
2.1 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

Artigos em revistas internacionais