Exploration of Using Generative AI to Solve CI Analytical Tasks
DOI:
https://doi.org/10.37380/jisib.v14.si1.2415Keywords:
competitive intelligence, generative AI, issues analysis, stakeholder analysis, trends analysisAbstract
This study examines the efficacy of generative AI platforms, including ChatGPT 3.5, Bing AI, and Bard, in performing competitive intelligence (CI) tasks such as Stakeholder Analysis, Issues Identification, and Trends Forecasting. These tasks are vital for analyzing various social, technological, economic, ecological, political, legal, ethical, and demographic factors that influence organizational or market dynamics. The main aim is to assess the capabilities of generative AI against human CI analysts in these areas, utilizing prompts and surveys to gather data.
The research engaged seven intelligence analysts of diverse experiences to evaluate the outputs from these AI platforms, establishing a benchmark for performance standards and identifying any limitations of generative AI in CI roles. The findings indicate that as of the end of 2023, generative AI does not match the performance level of human analysts in CI tasks. However, all platforms demonstrated some capacity to handle the tasks, with varying levels of success depending on the specific function, suggesting that while they cannot replace human analysts, they can augment their efforts.
Significantly, the study reveals that each AI platform has unique strengths and weaknesses, which were evident in their task-specific performances. Bard emerged as the most consistent, while Bing performed well in issues and stakeholder analysis but less so in trends forecasting. ChatGPT showed the least consistent results across the tasks.
The implications of these findings are profound for the integration of AI into CI roles. Analysts are encouraged to experiment with different AI tools to discover which best suits their needs, recognizing that these technologies are tools to assist rather than replace human analytical skills. This approach not only preserves but enhances the human element crucial in CI tasks, ensuring that AI and human intelligence together will shape the future of the competitive intelligence field.
References
Baidoo-Anu, D. and Ansah, L. (2023, September 12) Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning Journal of AI https://dergipark.org.tr/en/download/article-file/3307311 DOI: https://doi.org/10.2139/ssrn.4337484
Bass, D. (2023, January 23) 3.5 Bloomberg https://www.bloomberg.com/news/articles/2023-01-23/microsoft-makes-multibillion-dollar-investment-in-openai?embedded-checkout=true
Bensoussan, B. E., & Fleisher, C. S. (2012). Analysis without paralysis: 12 tools to make better strategic decisions. Ch. 13, Financial Times Press.
Cekuls, A. (2022) AI-Driven Competitive Intelligence: Enhancing Business Strategy and Decision Making Journal of Intelligence Studies in Business https://web-p-ebscohost-com.proxy.library.georgetown.edu/ehost/pdfviewer/pdfviewer?vid=0&sid=4326fb3b-011a-4dd7-8663-9e2b0b2b4363%40redis
Coulter, I, Elfenbaun, P., Jain, S and Jonas, W. (2016, January 7) SEaRCH™ expert panel process: streamlining the link between evidence and practice National Library of Medicine https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4704387/ DOI: https://doi.org/10.1186/s13104-015-1802-8
Emmanuel, C. (2023, August 3) GPT-3.5 and GPT-4 Comparison Medium https://medium.com/@chudeemmanuel3/gpt-3-5-and-gpt-4-comparison-47d837de2226#:~:text=With%201.5%20trillion%20parameters%20compared,structures%20and%20provide%20effective%20replies.
Faguy, A. (2023, March 21) Google Releases Bard AI Chatbot Amid Competition With ChatGPT Forbes https://www.forbes.com/sites/anafaguy/2023/03/21/google-releases-bard-ai-chatbot-amid-competition-with-chatgpt/?sh=5fe13bbe32b7
Feller, I. (2013) Handbook on the Theory and Practice of Program Evaluation, Edward Elgar Publishing https://books.google.com/books?hl=en&lr=&id=p4BQdisrW9kC&oi=fnd&pg=PA115&dq=benefits+of+an+expert+panel+for+qual+research&ots=jrFz1doqx7&sig=FBYFJVU-qqzRN-HUqU_3HuXAQ7g#v=onepage&q=benefits%20of%20an%20expert%20panel%20for%20qual%20research&f=false
Feuerriegel, S. Hartmann, J. Janiesch, C. & Zschech, P. (2023, September 12) Generative AI Springer https://link.springer.com/article/10.1007/s12599-023-00834-7 DOI: https://doi.org/10.2139/ssrn.4443189
Finkenstadt, D., Eapen, T., Sotiriadis, J., Guinto, P. (2023, November 30) Use GenAI to Improve Scenario Planning Harvard Business Review https://hbr.org/2023/11/use-genai-to-improve-scenario-planning DOI: https://doi.org/10.2139/ssrn.4760156
Fleisher, C. (2022) Trendsetting in competitive intelligence: Discovering your LIST Special Libraries Conference Annual Conference, Charlotte, NC.
Fleisher, A. & Tao, T. (2024, April 22) Human VS. Gen AI Analysis Competition SCIP https://www.scip.org/page/intellicon-us-nashville-program-2024#:~:text=Human%20VS.%20Gen%20AI%20Analysis%20Competition
Hargrave, M. (2023, May 11) Standard Deviation Formula and Uses vs. Variance Investopedia https://www.investopedia.com/terms/s/standarddeviation.asp
Hoffman, F. & Freyn, S. (2022, November 25) Competitive intelligence in an AI world: Practitioners’ thoughts on technological advances and the educational needs of their successors Journal of Intelligence Studies in Business https://web-p-ebscohost-com.proxy.library.georgetown.edu/ehost/pdfviewer/pdfviewer?vid=0&sid=9b1a4c2d-71b6-4965-9e41-f1304e1e9c54%40redis
Intel, (2017) Artificial Intelligence The Public Policy Opportunity Intel https://community.intel.com/legacyfs/online/files/Intel-Artificial-Intelligence-Public-Policy-White-Paper-2017.pdf
Joost N. Kok, Egbert J. W. Boers, Walter A. Kosters, and Peter van der Putten (2002) Artificial Intelligence: Definition, Trends, Techniques and Cases Encyclopedia of Life Support Systems https://www.eolss.net/Sample-Chapters/C15/E6-44.pdf
Khodyakov, D., Hempel, S. Rubenstein, L. Shekelle, P. Salem-Schatz, S. O’Neill, S. Danz, M. and Dalal, S. (2011, December 23) Conducting Online Expert panels: a feasibility and experimental replicability study Bio Med Central https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-11-174 DOI: https://doi.org/10.1186/1471-2288-11-174
Lim, W. Gunasekara, A. Pallant, J. L. Pallant J. I. Pechenkina, E. (2023, July) Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators Science Direct https://www.sciencedirect.com/science/article/pii/S1472811723000289#bib42 DOI: https://doi.org/10.1016/j.ijme.2023.100790
Mannuru, N. R., Shahriar, S., Teel, Z. A., Wang, T., Lund, B. D., Tijani, S., Pohboon, C. O., Agbaji, D., Alhassan, J., Galley, J., Kousari, R., Ogbadu-Oladapo, L., Saurav, S. K., Srivastava, A., Tummuru, S. P., Uppala, S., & Vaidya, P. (2023). Artificial intelligence in developing countries: The impact of generative artificial intelligence (AI) technologies for development Information Development https://www.researchgate.net/profile/Lydia-Oladapo-3/publication/373934747_Artificial_intelligence_in_developing_countries_The_impact_of_generative_artificial_intelligence_AI_technologies_for_development/links/6504d6dea69a4e63180fcb33/Artificial-intelligence-in-developing-countries-The-impact-of-generative-artificial-intelligence-AI-technologies-for-development.pdf DOI: https://doi.org/10.1177/02666669231200628
Marrm B. (2023, August 16) What Does Generative AI Mean For Websites And SEO? Forbes https://www.forbes.com/sites/bernardmarr/2023/08/15/what-does-generative-ai-mean-for-websites-and-seo/?sh=f416f692af94
Martinez, R. (2019, May 28) Artificial Intelligence: Distinguishing Between Types and Definitions Nevada Law Journal https://scholars.law.unlv.edu/cgi/viewcontent.cgi?article=1799&context=nlj
McCarthy, J. (2004, Nov. 12) What is Artificial Intelligence? Stanford https://www-formal.stanford.edu/jmc/whatisai.pdf
Mehdi, Y. (2023, May 4) Announcing the next wave of AI innovation with Microsoft Bing and Edge Microsoft https://blogs.microsoft.com/blog/2023/05/04/announcing-the-next-wave-of-ai-innovation-with-microsoft-bing-and-edge/
McGill, (2023) Qualitative or Quantitative Research? McGill Qualitative Health Research Group https://www.mcgill.ca/mqhrg/resources/what-difference-between-qualitative-and-quantitative-research
Microsoft, (2024, April 2) AI features for Teams and Classic copilots Learn Microsoft https://learn.microsoft.com/en-us/microsoft-copilot-studio/advanced-ai-features
Mok A. & Zinkula J. (2023, June 4) ChatGPT may be coming for our jobs. Here are the 10 roles that AI is most likely to replace Business Insider https://www.businessinsider.com/chatgpt-jobs-at-risk-replacement-artificial-intelligence-ai-labor-trends-2023-02
Narang, S., Chowdhery, A., Google Research Engineers, (2023, April 4) Pathways Language Model (PaLM): Scaling to 540 Billion Parameters for Breakthrough Performance Google Research https://blog.research.google/2022/04/pathways-language-model-palm-scaling-to.html
OpenAI, (2022, November 30) Introducing ChatGPT OpenAI https://openai.com/blog/chatgpt
OpenAI, (2023, March 14) GPT-4 OpenAI https://openai.com/research/gpt-4
OpenAI, (2023) Fine Tuning OpenAI https://platform.openai.com/docs/guides/fine-tuning
OpenAI, (2024) ChatGPT – Release Notes OpenAI https://help.openai.com/en/articles/6825453-chatgpt-release-notes
Pasichnyk, O & Strelkova, O. (2022) Three Types of Artificial Intelligence Khmelnitsky National University http://eztuir.ztu.edu.ua/bitstream/handle/123456789/6479/142.pdf?sequence=1&i
Peres, R. Schrier, M. Schweidel, D. Soescu, A. (2023, June 2) On ChatGPT and beyond: How generative artificial intelligence may affect research, teaching, and practice Science Direct https://www.sciencedirect.com/science/article/pii/S0167811623000162 DOI: https://doi.org/10.1016/j.ijresmar.2023.03.001
Pichai, S. & Hassibis, D. (2023, December 6) Introducing Gemini: our largest and most capable AI model Google https://blog.google/technology/ai/google-gemini-ai/#sundar-note
Pichal, S. (2023, February 6) An important next step on our AI journey Google https://blog.google/technology/ai/bard-google-ai-search-updates/
Rajaraman, V. (2014, April 30) John McCarthy — Father of artificial intelligence Springer https://link.springer.com/article/10.1007/s12045-014-0027-9
Reimann, N. (2023, January 5) ChatGPT Creator OpenAI Discussing Offer Valuing Company At $29 Billion, Report Says Forbes https://www.forbes.com/sites/nicholasreimann/2023/01/05/chatgpt-creator-openai-discussing-offer-valuing-company-at-29-billion-report-says/?sh=7b4563be11e0
Russell, & Norvigg, (2022, Aug. 22) Artificial Intelligence: A Modern Approach, 4th US ed Pearson https://aima.cs.berkeley.edu/
Sahoo, S. Kumar, S. Abedin, M. Lim, W. Jakhar, S. (2023, January 27) Deep learning applications in manufacturing operations: a review of trends and ways forward Journal of Enterprise Information Management https://www.emerald.com/insight/content/doi/10.1108/JEIM-01-2022-0025/full/html
Tran, H & Murphy, P. (2023, September 1) Generative artificial intelligence and entrepreneurial performance Emerald Insight https://www.emerald.com/insight/content/doi/10.1108/JSBED-09-2023-508/full/html
Verma, P. & De Vynck, G. (2023, June 2) ChatGPT took their jobs. Now they walk dogs and fix air conditioners. Washington Post https://www.washingtonpost.com/technology/2023/06/02/ai-taking-jobs/
Wang, P. (2019, August 12) On Defining Artificial Intelligence Sciendo https://sciendo.com/downloadpdf/journals/jagi/10/2/article-p1.pdf
Wankhede, C. (2023, December 7) What is PaLM 2: Google's large language model explained Android Authority https://www.androidauthority.com/what-is-google-palm-2-3331329/
Wodecki, B. (2022, July 27) 7 Language Models You Need to Know AI Business https://aibusiness.com/nlp/7-language-models-you-need-to-know
Zhang, P. & Boulos, M. (2023, August 24) Generative AI in Medicine and Healthcare: Promises, Opportunities and Challenges MDPI https://www.mdpi.com/1999-5903/15/9/286. DOI: https://doi.org/10.3390/fi15090286
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Journal of Intelligence Studies in Business
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
