A Visual Language for Humans 👀

Recent discussions about UI/UX design have placed a more critical view on flat design (Norman & Tognazzini, 2015) and how the departure from the visual affordances found in skeuomorphism favors aesthetics rather than functionality. From a user’s perspective, these debates can be experienced when focusing on the usability of an interface, i.e. which App or Operating System is easier to use. Nonetheless, usability is also dependent on the user’s ability to read and understand the visual language of computers. In turn, the use of these affordances is affected by the user’s memory and intelligence. In order to explore this relation, I along with my team actively observe and decode the signifiers of the Apps we use the most. The goal is that we find out what is the visual language of the interface and how it enables us to achieve your goal.

What was this project about and what my team and I did?

• We explored how affordance, memory, and intelligence play role in the experience of users in different contexts and settings.

• We researched affordance, memory, and intelligence to a practical activity we face and propose solutions through HCI design guidelines.

• We created a preliminary Design Aid Device (DeDe) incorporating specific evidence to address the impacts of affordance, memory, and intelligence on the HCI.

How can we design interactive systems by using a visual language that is rooted in memory, intelligence, and affordances?

1. App & platform choice and rationale

   We chose to analyze with many social media apps on Mobile, but we mainly focus on Facebook, Instagram, and Reddit, at the end of our research, we apply what we observed to make changes to Instagram’s layout, and icon shape. We thought Instagram and Facebook are the two massive social media app with more than 1 billion downloads, and they are both from the same company, so they must have some design decisions that make the app satisfies a large number of users.

2. Definitions

   Affordance: Generally speaking, affordance is a relationship between an object and a person, determining how the object can be used (Norman, 2013). Besides, we also have hidden affordance which forms through experience, trial and error, and false affordance which suggests you can you something, but you actually can’t (Borowska, 2017). Affordance visibility is important to let us know how to use an object correctly and avoid misuse of the object. The same thing for icons and gestures in our devices (digital affordance), we don’t want to design any obscure or misleading icons or gestures that result in unwanted action.
   
   

   Memory: Consists of working memory (short-term), semantic memory (store knowledge and facts), and most importantly, episodic memory (store episodes of our life) which consists of encoding, storage, and retrieval, and they are the three main stages of processing memory (Roediger & McDermott, 2021). Our brain is better at recognizing things than recalling them, that’s why most functions on many interfaces are usually shown to the user, allow them to select instead of manually recalling the function name and type it out (although sophisticated expression can be done by this way) (Johnson, 2010).
   
   Intelligence: The intellectual ability that consists of the ability to learn, remember, and use new information to solve problems adaptively. According to Horn and Cattell, there are two types of intelligence: “Fluid intelligence” which allows reflexive problem solving, and “Crystalized intelligence”, which accumulates from learning and experiencing (Horn & Cattel, 1966). “IQ” or “intelligence quotient” is a name given to the score of the Binet-Simon test. The score is derived by dividing a child’s mental age (the score from the test) by their chronological age to create an overall quotient (Biswas-Diener, 2021).

3. Brainstorming

   • What is known & unknown

     • Known

       • User experience influences how they learn to use new applications.

       • Hidden affordance can be created through experiencing with similar interface, and through trial and error.

       • . Icons with a clear signifier would allow users to guess its function better when pressed on it • Users don’t look at the whole screen, they only scan for things that seem to pop out of the screen (Large text, colorful area).

       • Dark patterns are evil, yet profitable, but still evil.

       • Most companies use the scrolling system to get their users hooked onto their app • The likes, comments, mentions give us small doses of dopamine boost which makes us check our phone constantly to see how many new likes, comments, or mentions we got.

     • Unknown

       • What percentage of people prefer the swiping mechanism rather than pressing on each tab. • How users would react to changes when they are already used to the interface. Once people are used to something, no matter if the process is simplified, they’ll be annoyed by change. • Are there any real-life shapes that can’t be turned into iconic affordance?

       • Are there any specific color schemes for interfaces that would make it hard for people with color blindness to see?

   • Design objective and research question

     • Reorganize Instagram buttons, categorize information, and make its navigating gestures and icons similar to other apps.

   • Individual Research Questions for Research:

     • Is Instagram button placement optimized for user experience or revenue?

     • How to maintain gesture consistency when navigating through tabs on Facebook and Instagram?

     • Do icons’ consistency and animations provide users with better feedback on which tab they are in?

     • Provide iconic, echoic, and haptic feedback for a better understanding of when a post is posted or not.

     • The activity tab seems disorganized, how to save users’ time in that tab?

Mindmap

4. Research Knowledge

Research point 1: Button placement seems unconventional - looking for patterns from other apps to  create a unified mental model (Memory & Affordance)

As we did our investigation on other social media platforms, we found some similarities in their button  layouts:

1. Important tabs/features are made visible and accessible by placing them in the main navigation bar.

2. The upload/create button which generates content is usually placed in the middle of the navigation bar.

3. The search bar is always on the top so that results can always drop down instead of dropping up.

• Problem: Instead of following these layouts, Instagram placed “Reel” and “Shop” tabs in the middle of their navigation bar, making them more visible to users, promoting those new functions. (Design book ->  page 3 last line) Because we tap buttons without looking at them, we expect things based on our experience rather than what is actually on the screen, Instagram successfully tricks us into “trying out”  their new features.

• Solution: Moving buttons and tabs to make it look similar to most apps, reduce confusion.
  
  

Research point 2: The activity tab is cluttered - categorization would improve info scanning (Vision). Many people don’t read word-by-word when presented with text on an interface, even if the  information is brand new to them (Nielsen, 1997)

• Problem: Instagram’s activity tab has all kind of information in it; thus, users have to spend more time  scanning more words to understand what each entry is about

• Solution: Categorizing activities, put them into different tabs so that users will implicitly know what they are reading in each tab without spending too much time.
  
  

Research point 3: Instagram’s navigating gesture (swipe left/right to switch tab) does not match its navigation bar hidden affordance - Make it similar to other apps (Memory & Affordance & Signifier). An affordance is hidden if it is based on users’ experience with similar interfaces (Borowska, 2017). For people who have been using social platforms like Facebook and Snapchat a lot, they would know which tab they are at, and they know they can go to the next/previous tab by swiping left/right as implicitly indicated by the navigation bar and highlighted icon.

• Problem: This doesn’t apply to Instagram as swiping right will bring up a hidden camera tab and swiping left will bring you to the “Message” tab. Frequent Instagram users would know about this, but people who use multiple social platforms will find this confusing.

• Solution: Enabling swiping left/right to navigate through tabs on Instagram to unify the navigating gesture with other apps.

Research point 4: Instagram’s “Message” airplane-shaped icon is different from most apps’ “Message”  speech-bubble-shaped icon – Change the icon to the bubble shape (Affordance). Another type of affordance we discovered was iconic (metaphorical) affordance, which uses shapes in real life or shapes that we are familiar with to create icons that relate to real-life objects’ function, for example, envelope email icon affords to send an email (WebdesignerDepot, 2015). For digital affordance, the speech bubble shape affords their messaging function, and this is used widely in many apps. Meanwhile, Instagram “Message” icon is in the shape of a paper airplane (to me it looks just like 2  triangles).

• Problem: Users coming from other apps may have difficulties finding the messaging function because of the unconventional icon Instagram is using.

• Solution: Change Instagram’s “Message” icon into a speech bubble shape, like Facebook Messenger

5. My Experience

Day 1:

I have observed 2 apps for this assignment - Facebook and Reddit. I have logged onto  Facebook after a very long time and the UI has changed so much that I was having a bit of difficulty at navigating first. I downloaded Reddit and signed up and this was my first time using the app and the UI seemed a bit complicated but still manageable.

Day 2:

Today I realized facebook moved their messenger icon on the top-right, which at first seemed quite weird. As I started to navigate more, I found myself being hooked more on the “reel” tab where there are continuous videos playing on my preference. I figured out that Reddit has swipe options to change the home page and has a navigation bar on the bottom which makes complete sense.

Day 3:

I kind of realized that Facebook has intentionally placed the reel tab on the bottom because that is where I would find myself sliding more and get hooked with the videos and I would be spending countless hours consuming media. The messenger icon would hide on the top so that I do not leave the Facebook app. Today I learned that the Reddit app replaced the hamburger menu on the left with its icon and when pressed, it prompts with more information such as settings.

Day 4:

Facebook’s navigation seems more seamless than ever and today when I clicked on the hamburger menu, it led me to the settings page with most of the options in just one page which is amazing because I do not have to keep navigating settings and sub-settings. I learned what points mean in Reddit and how to use it more effectively, and that I can use it as a tool to my advantage.

Day 5:

I feel like both Facebook and Reddit have been designed in such a way that helps their user understand the app really quickly, by adapting the rules of affordance, memory,  and attention in their favor. Because both the apps use the navigation bar, I have quickly stored it in my memory, and every time I open the apps the surroundings of the app indicate which icon will I be next clicking. Both of them have their search bars on the top and home menu on the bottom. They did a great job at grabbing my attention to things they wanted to.

Our Findings

1. Explanation:

   • Short Version:

     • Problems and solution:

       • Icon/tab placements that promote revenue purposes rather than following other apps layouts to satisfy users’ experience -> Solution: Move frequently used tabs to the main navigation bar, and move promoted tabs to other places.

       • Misleading signifier and gestures when compared to Facebook or other apps i.e., users thought they could swipe through tabs, but they actually can’t -> Solution: allowing swiping through tabs like Facebook.

       • Unorganized activity tab with unrelated entries -> Solution: Categorize entries to make allow faster information scanning.

       • Instagram’s messaging icon is different from most apps -> Solution: Change it from a paper airplane shape to a speech bubble.

Long Version:

• Instagram and Facebook are two sibling social media platforms. Users who are familiar with either interface should be able to use the others at ease since most gestures and icons’ positioning are quite similar. However, although the navigation bar of both apps looks almost identical, the navigating gesture is a bit different, and the representation of Instagram's navigation bar is somewhat misleading and diverted from the common layout compared to Facebook or most social apps.

• To be more specific, on Facebook, swiping left/right would bring you to the left/right tab. But swiping left on Instagram would bring you to the camera, and swiping right would bring you to the message tab. Additionally, So, we would like to change this distinctive gesture of Instagram into the  gesture that users are familiar with in many apps so that when they see the navigation bar (the memory  cue), they know that they can swipe left and right to navigate through the tabs

• Moreover, the placement of tabs and buttons on Instagram seems to promote “Instagram Shop”  and “Instagram Reel” instead of placing things in the right place to satisfy the user experience. To clarify this, for most users, we use the top right tabs (Upload, Activity, Message) much more often than the  “Reel” and “Shop” tab. Thus, we wanted to swap the “Reel” and “Shop” tabs with the top right corner tabs.

• Besides, we would also put the “Upload” icon at the center of the navigation bar, because users’  gaze is usually positioned in the middle of the screen, so putting the “Upload” icon at the center of the navigation bar would make it easier to find since it is the main and most important function of  Instagram. To add up, most apps’ main function is placed at the center, so once again, putting the  “Upload” tab in the center would align the layout with users’ experience on other apps.

Thirdly, most social app users are familiar with the bubble-shaped messaging icon. However, on Instagram, its messaging icon is in the shape of a paper airplane. New Instagram users might not even acknowledge the existence of this function. Thus, we want to change this icon into a speech bubble, to make it fit with the conventional icon, and apply iconic affordance to the button.

- Here is the result of our proposed icon placement and shape changes.

Lastly, we would like to categorize activities in the “Activity” tab just like how Facebook does. We found that entries in the “Activity” tab are unorganized, un-relatable, and serves as a promotion for interaction on Instagram. Users usually do not read the whole text but just look for keywords to understand what an entry means, so if we put entries in different categories, the user would implicitly understand what they are reading while scanning for fewer words.

2. Effect:

• If we can reposition icons, change swiping gesture, change the messaging icon, and group  activity entries to make Instagram look more like Facebook and other apps, then the user would not  confuse themselves when navigating through the app and save time finding and scanning for the  information they need because these changes would make it look like other social apps, allowing users to transfer their experience with other social platforms to Instagram

3. Upsides:

• The new messaging icon will inform users of the existence of the function • Made the app behave more like its Facebook as well as other social platforms like Reddit and  YouTube so that user won’t get confused when they apply their experience on Instagram • By making frequently used functions more visible and accessible, we can save users’ effort to look for the things they need  

• Condensed information in the activity tab, save time reading each activity entry 4.

4. Downsides:

• Users’ retention on Instagram decreases as the app cannot hook the user in with the  captivatingly placed “Reel” tab,  

• Might cause some revenue loss for Instagram because be “Shop” is moved to the top

• The new “Upload” icon can be distracting for lurkers (people who prefer surfing to upload)

5. Context.

• Screen size is an important factor for this DeDe. Although we move the “Reel” and “Shop” tabs to the top right corner, they are still in the users’ gaze if they are using phones or devices with small screens. But for tablets and devices with a large screen, those tabs might be out of users’ sight,  making it much less noticeable.

• As layout always get updated when new functions are introduced, there will be a time when our  proposed layout and gesture might become unfitting with the purpose of Instagram

Reference

Norman, D. A. (2013). The Design of Everyday Things. New York: The Perseus Books Group.

Borowska, P. (2017). Understanding affordance in digital interfaces " Paul Olyslager. Retrieved  March 11, 2021, from https://www.paulolyslager.com/understanding-affordance-digital interfaces/

McDermott, K. B. & Roediger, H. L. (2021). Memory (encoding, storage, retrieval). In R.  Biswas-Diener & E. Diener (Eds), Noba textbook series: Psychology. Champaign, IL: DEF  publishers. Retrieved from http://noba.to/bdc4uger 

Johnson, J. (2010). Designing with the mind in mind : Simple guide to understanding user  interface design rules. ProQuest Ebook Central https://ebookcentral-proquest com.proxy.lib.sfu.ca 

Horn, J. L., & Cattell, R. B. (1966). Refinement and test of the theory of fluid and crystallized  general intelligences. Journal of Educational Psychology, 57(5), 253-270

Biswas-Diener, R. (2021). Intelligence. In R. Biswas-Diener & E. Diener (Eds), Noba textbook  series: Psychology. Champaign, IL: DEF publishers. Retrieved from http://noba.to/ncb2h79v 

Nielsen, J. (1997). How users read on the web. Retrieved March 12, 2021, from https://www.nngroup.com/articles/how-users-read-on-the-web/

WebdesignerDepot. (2015). How to perfect ux with design affordances. Retrieved March 12, 2021, from https://medium.com/@WebdesignerDepot/how-to-perfect-ux-with-design affordances-79ce78792b08