Software was sent through a simulated workplace chat system to mimic a typical office scenario. Each participant received six files per session, half of which were genuine tools and half recompiled samples of ransomware, remote access trojans, or cryptocurrency miners. None of the files were labelled, and participants were free to search online or use built-in tools to investigate.
First Session Without Extra Tools
In the initial round, users relied only on what came with the operating system or could be found on the internet. Malware was correctly identified 88 percent of the time, but safe software was mistaken for harmful in many cases, leading to a lower 62 percent accuracy rate for benign files. Most decisions took around four to five minutes, though faster judgments often had more errors.
Analysis of participant notes revealed four main types of indicators: details about the installation file, the program’s behaviour after running, its visual design, and information from outside sources. Within these categories, 25 more specific factors were identified, such as file signatures, CPU usage, network destinations, and online search results.
Second Session With Enhanced Process Data
The second round introduced a custom-built task manager showing extra process details, including resource use, files accessed, and the country linked to any network connections. No guidance was given on how to interpret the information. With this additional view, malware detection improved to 94 percent and overall accuracy rose to 80 percent. Correct classification of benign programs improved slightly but remained below malware accuracy.
Participants who already understood system metrics used the tool to confirm suspicions quickly. Less experienced users found the added information useful but sometimes struggled to judge what it meant. The enhanced display of network destinations, in particular, led many to question software that connected to unexpected countries.
Indicators That Shape Judgments
Advanced participants relied on technical signs like resource load or suspicious network activity. Basic users paid more attention to file names, icons, and the overall look of the software. Misunderstandings were common. Some believed that a neat interface meant a program was safe. Others assumed the shield icon on an installer confirmed safety, when it actually indicated a request for higher privileges.
False positives were often caused by outdated designs, unfamiliar publishers, or installer behaviour that appeared unusual but was harmless. The same feature could push one person toward suspicion and another toward trust, showing how personal experience influences decisions.
Implications for System Design
The study highlights that while many people can spot harmful software in realistic conditions, safe programs are more likely to be flagged as threats. Small improvements to operating system tools, such as clearer displays of file activity and network connections, could help users make better decisions.
By collecting more than 2,600 real-time observations, the research offers insight into how people investigate unknown files. It shows that improving the clarity and accessibility of key indicators may raise detection accuracy without requiring advanced technical training.
Notes: This post was edited/created using GenAI tools. Image: DIW-Aigen.
Read next: Musk Scores Lowest U.S. Favorability in Gallup Poll, Netanyahu Confronts Gaza Genocide Allegations
