Ok so for this assignment I’m going to test Keenious and Perplexity Deep search on the task which I found to be the most annoying part of RHET 1020: finding and extracting information from research papers. The method with which I’ll assess the two tools through is that I’ll ask each tool to find research papers that would have been relevant to my RHET 1020 research paper topic, and see if they can provide me with articles that are as relevant and high quality as the ones I ended up choosing for my 1020 paper. For the second part of the assessment, I will ask each AI tool to summarize, explain, and extract information from a paper that I wrote, so I will be able to assess how well they understood my paper and whether or not they were able to extract the important pieces of information.

Task 1: Finding Scholarly Articles

So, I gave each of the AI tools the same prompt:

“I have a research paper where the task is that I have to thoroughly explain a problem in the world. The issue I chose to examine was how colonialism has effected the development of African countries. To analyze this issue, I focused particularly on the region known as the ‘CFA zone’ in West Africa. My goal is to find research papers that talk about how France’s influence in the CFA zone, their control of monetary policies, as well as all their other means of soft power in West Africa has negatively impacted the development of these countries. Can you find me some high quality scholarly articles (not books) on the subject?”

Here were the results:

Keenious results:

In this photo you can see the first 5 articles that Keenious found for me based on my prompt

Heere you can see the last 5 articles Keenious found for me, meaning it found 10 articles in total.

 

Perplexity Results:

In this image you can see the first three articles perplexity found

Here you can see another 3 articles perplexity found

Here you can see the 7th and final article perplexity found

 

Analysis of Task 1: 

Before I continue, let me just say I am using the bog-standard free version of both services. I made a free account with each tool and used the research based tool for each. In the case of perplexity, I used the deep search feature and in the case of Keenious I chose the feature “Keenious for researchers” and then fed my prompt to each. As you can see from the results both came up with multiple sources, Perplexity found 7 and Keenious found 10. In the case of Keenious, of the 10 articles it suggested I actually was already familiar with 6 of them as I had consulted and or cited them in my work last semester. So Keenious was able to provide high quality sources which were peer reviewed and on topic, so the instructions of my prompt were followed properly. Similarly, Perplexity provided 2 articles which I read and used last semester, and as a matter of fact these two articles are found in both AI searches. Perplexity also followed my instructions, just as well as Keenious. So in this task, I’d consider them equal. However, if I had to choose one I would choose Perplexity. The reason is that in my prompt, its very evident that I am arguing for a particular interpretation of the situation in West Africa. I clearly state that I want papers that explain how France has negatively impacted the region’s development. The main difference between the results in Perplexity’s search and Keenious’ is that Perplexity got me articles that all agree with the underlying statement in my prompt. All of the articles paint France as a neocolonial power, leveraging the institutions it established in the region during the colonial era to disproportionately benefit France in the post-colonial era. This is very valuable to my research as part of the RHET 1020 assignment was to assert why your topic is a problem. Now, some professors may tell you that conducting research in this manner, with a pre-determined result and in search of a particular interpretation of the literature, is wrong and that the fact that Keenious provided me with articles that may challenge my position is actually a plus. I disagree with that sentiment though, as all research is inherently biased. I would rather present two sides of a debate by showing examples of articles that are fully transparent in showing which side of the fence they fall on. So in this task, Perplexity is the winner by the slightest of margins.

Task 2: Analyzing articles

So for this task I actually uploaded the first part of my research paper on the subject which I prompted the AI to research, but to ensure that initial prompt couldn’t have had any effect I made sure to ask this new prompt on a new account. Just like before, I fed each the exact same prompt:

“Please analyze this article and tell me what is the main argument the author is making, and what are the ways in which he supports these arguments? What are the most important pieces of information from this article if I want to write an article on the same topic?”

Keenious Response:

This is a photo of Keenious' first response to my prompt, where it very very very briefly summarizes my article (in 3 sentences) and then says it can not help suggest how this article could help me in my own writing.

Here keenious finally begins to analyze my article, this is the first halfof its response

 

This is the 2nd half of the Keenious analysis

 

Perplexity Response:

This image shows the first part of the Perplexity analysis of my paper

 

Here is the 2nd part of perplexity's anaylsis

Here is the 3rd part of perplexity's analysis

Here is the fourth part of perplexity's response

Here is the 5th and final part of perplexity's anaylsis

 

Task 2 Analysis:

In this task, there is no comparison to be made. Firstly, Keenious denied my request to analyze my paper when I was using the “Keenious for researchers” feature. It said that this paper is a student paper so its not worth analyzing, then I explained to it that I am specifically testing the AI’s capability to analyze articles by giving it a paper to analyze, the contents of which I am intimately aware of and so I would be able to easily assess the AI’s ability to analyze papers. Even after that explanation, it refused my request to analyze the paper (I should have screenshotted this interaction before closing the tab) on the basis of it being a research tool, but I would assume analyzing papers is a very important feature for a research tool but hey I’m no developer. So I opened a new tab and a new account (to ensure that previous prompts don’t effect the response of the AI) and I uploaded my paper to the normal Keenious AI and asked it the prompt above. Then it analyzed and gave me a super brief analysis of the paper. So I had to prompt the AI to expand and give me a more detailed analysis (which I show in the photo), and finally it gave the response which I provided in the photos. The Keenious response was all accurate, there were no hallucinations, and it responded to my prompt adequately. In the case of Perplexity, it immediately analyzed my paper with no questions asked or having to prompt it again. It did not make any mention of the fact this paper is a student paper, which could be problematic if this article were poorly researched and done haphazardly, but luckily this article is an exquisite piece of research which received an A- grade and was lauded by its professor, so take that Keenious. Perplexity’s analysis of my paper was much more detailed in depth, and it went above and beyond what I expected. So in this task, Perplexity was significantly more useful.

 

Conclusion: 

I prefer perplexity, its analysis was superior to Keenious by a distinct margin, and its research was just as good as Keenious (and in my opinion, slightly better).