New Microsoft Teams Performance Benchmarkv1.0

1. Executive Summary

This GigaOm Benchmark report was commissioned by Microsoft.

Microsoft Teams (Teams) is a collaboration platform that combines workplace chat, video meetings, file storage, and application integration. It is part of the Microsoft 365 suite of applications and is designed to help teams stay organized and connected. Teams allows users to communicate and collaborate in real-time, share files, and access applications from a single platform. As application performance is key to productivity, Teams has a regular cadence of releases focused on enhancements to functionality, performance, and ease of use.

This report benchmarks the performance differences between two versions of the Microsoft Teams desktop application for Windows: classic Teams (version 1.1, release June 2022) and new Teams (version 2.1, preview release March 2023). It showcases the improvements provided by new Teams by comparing the results of tests across three domains—installation behavior, application responsiveness, and resource utilization. Our findings:

• Installation behavior: Up to 3x faster time to install
• Application responsiveness: Up to 2x faster launching Teams, 2.5x faster joining meetings, and 1.7x faster loading chats and channels
• Resource utilization: Overall 50% reduction in our tests of disk and memory resource consumption

New Teams outperformed classic Teams in all tests. Our timed tests showed new Teams across all three testbeds to be on average 2x faster than classic Teams when starting up, joining meetings, and switching chats and channels. Some of the results were remarkable. Join time performance more than tripled for our high-end system, while general purpose and low-end systems saw 2.5x and 2x improvements, respectively. We saw similar 2x to 3x improvements in application launch times for our two higher-end platforms, as well as a nearly 70% reduction in disk space consumption.

Our analysis of typical usage patterns shows that businesses can expect employees to save up to five minutes per user per week when using new Teams vs. classic Teams (Figure 1). We then applied these gains across three typical organization types to gauge aggregate impacts: a 250-employee small-to-medium business (SMB), a 2,000-employee midsize enterprise, and a 55,000-employee large enterprise.

Figure 1. Aggregate Minutes Spent per Employee per Week (lower is better)

2. New Microsoft Teams

New Microsoft Teams is a ground-up platform overhaul focused on performance improvements across all aspects, improving application responsiveness, fluid interaction, and resource consumption. Microsoft worked to reduce the time required for users to connect and reduced consumption of device resources like memory and disk space, while enhancing multitenant collaboration and the ability to use multiple accounts in Teams simultaneously. The changes also focused on better manageability through alignment with Microsoft solutions to improve installation reliability and reduce network bandwidth and disk space consumption.

As customers try out the new Teams, they will likely encounter some gaps as this preview release provides a subset of the features available in classic Teams. Features yet to be released include third-party apps, Line of Business (LOB) apps, advanced calling features such as call queues, and advanced meeting capabilities, including breakout rooms.

This report focuses on measuring the performance improvements most valuable to end users, including time to launch the application, time to join a meeting, and delays in task and context switching. We also measured disk and memory resource consumption, which impact overall system performance and stability. While the gross impacts in terms of time saved per meeting are small (measured in seconds per instance for individual users), the time lost adds up over a period of weeks and across a large user base.

More to the point, these gains in responsiveness enable a fluid, productive user environment that minimizes distraction, disruption, and delay. The ability to make interaction with Teams more seamless and to reduce the “weight” of task switching not only has direct value in time saved, it reduces the drain on productivity from the task switching itself. The new Teams is nearly twice as fast across the actions we measured, and each improvement represents a moment when a user is not waiting for a machine to complete a task. (See Table 1. Annual Time Savings Using New Teams)

3. Scenarios

Our goal in testing was to simulate relatable, “real-life” user experiences in both classic Teams and new Teams. To do so, we selected three different devices that map closely to different personas within an organization.

We selected a low-end device to represent the usage of workers in sectors like education and manufacturing, which typically work with less-expensive systems, as well as the cohort of aging machines in all organizations that often lack sufficient RAM or processing power. Users on low-end systems often leverage Teams to chat or collaborate with other workers and managers.

The general purpose laptop represents the vast majority of users who may interact with Teams in various ways. They may be in the office, remote, or traveling to other cities and countries. This persona also leverages Teams on multiple platforms such as mobile and tablet in addition to the laptop. They may even need to switch between devices on the go.

The high-performance device represents a power user. This can be a developer or a power user in an organization’s finance department. These users access many parts of Teams simultaneously. They may be on a video call, sharing files, and chatting with team members simultaneously. They likely operate multiple channels during a typical day. They may be working on several different projects at the same time with numerous parts of the organization. They are probably on/off many Teams meetings throughout the day. The teams they interact with may be onsite, remote, and/or distributed worldwide. They may multi-task and use Teams when using other applications such as developer tools or financial systems. They often have a more powerful device to handle the workload both in and outside Teams.

Each of these users is impacted by Teams. The solution drives productivity, increased collaboration, and alignment for organizations. If Teams performs faster and easier, each role will experience increased productivity and time savings.

Each test was performed five times on each version of Teams on each device, with the results recorded and measured using standard and consistent methods. For those tests that required timing of actions, video was used, and a frame-by-frame analysis performed to capture events with sub-second resolution. These results were then rounded to the nearest second. The data was inspected for anomalies and averaged to provide a result.

4. Environment: Configuration and Setup

One of the primary goals of GigaOm Benchmark reports is to make the tests easy for users to reproduce. In this section, we describe the elements and processes necessary to perform the tests or change the variables according to the user’s needs. In the benchmark, we aimed to create environments as close to an apples-to-apples comparison as possible.

• Application/Products
  • Microsoft new Teams (version 2.1, preview release March 2023)
  • Microsoft classic Teams (version 1.1, released June 2022)
• Test devices
  • Laptop A – Low End: Lenovo IdeaPad 1 – 15.6″ Laptop (2023 Model), AMD Dual-core Athlon 3050U Processor, 4GB RAM, 128GB SSD with Windows 11
  • Laptop B – General Purpose: Microsoft Surface Laptop 4 – 13.5″ touch screen, Intel Core i5-1145G7 4-core CPU, 8GB RAM, 512GB SSD with Windows 11 OS
  • Laptop C – High End: ASUS Zenbook Pro Duo 15 OLED – 15.6″ home and business laptop, Intel i9-12900H 14-core CPU, 32GB LPDDr5 4800MHz RAM, 1TB SSD with Windows 11 Pro OS
• Microsoft 365 Tenant (required for the tested pre-release version of new Teams)
  • Sample tenant configured by Microsoft
  • 20 users configured to defaults of a “trial tenant”
  • Users provisioned for both classic Teams and new Teams
  • Meetings created by Gigaom using Gigaom participants with no coordination from Microsoft
  • All data used during the tests was generated by Gigaom for the purpose of testing Teams performance. Gigaom attempted to recreate normal Teams usage through text, graphics, animated GIFs, and video in channels.

Tools used:

• Process Explorer used to measure memory allocation
• Task Manager used to show memory usage
• Video recordings of the experience used to measure time differences between the two versions of Teams
• Microsoft Explorer used to examine disk usage
• Video editing software used for precise measurements on each timed test

5. Tests

The following tests were conducted comparing classic Teams to new Teams:

Installation Behavior

• Installation time: Time required to install each version of Teams from an initial state
• Installation disk space: Disk space consumed after installation

We used a digital device to record the installation of classic Teams and new Teams on each device. We used video editing software to measure the install time. Leveraging this software, we could measure the installation time to a millisecond.

Application Responsiveness and Fluidity

• Time required to launch the application
• Time required to join a meeting from a Teams calendar link
• Time required for a user to switch between chats
  • With text only
  • With text and images
• Time required for a user to switch between channels

Application responsiveness can be difficult to benchmark due to the manual process. By recording the tests, then importing them into video editing software, they can be stopped, trimmed, and analyzed for frame-specific start and stop times, giving a very accurate (to the sub-second) measure of time.

Resource Utilization

In addition to the above tests, GigaOm captured system state and telemetry regarding:

• System memory consumption
• Disk space utilization

To gauge each laptop’s subsystem, we used a tool from Microsoft’s own library called Process Explorer from the Microsoft Sysinternals Suite. Focusing on the process tab, we could measure memory utilization and impacts on other subsystems.

6. Results

Overall, new Teams showed significant improvement over classic Teams in all tested categories. Our testers felt the new Teams version was easier to use, faster to load, and provided a better user experience overall. The underlying subsystems, specifically system memory and disk storage, were more efficiently used even on the lower-end device.

Results are based on data from functional and non-functional testing methods. We ran each test five times, and analyzed the data with three statistical models. First, we took an average of the timings for each test and each device. This gave us the raw data from each version and device without accounting for outliers or anomalies. In looking at the data, there were a few outliers in the results. We then normalized the results in two different ways. We calculated the mean values by discarding the highest and lowest values, then averaged the remaining results. We also calculated the quartile value, discarded any values not within the quartile, and then averaged the results. Ultimately, we believe the quartile average results provide the most accurate picture of the testing results and experience.

Observations on the Data

While the tests we ran generally addressed day-to-day interaction with the Teams application, we did opt to test the time required to install the two versions of Teams on each of our testbed laptops. We discovered steep reductions in time on task, as shown in Figure 2, producing a 3.4x advantage in time to install for new Teams across our three testbeds. Breaking it down: Installation was 1.8x quicker on our general purpose laptop, 2.8x quicker on our high-end laptop , and a remarkable 4.4x quicker on our low-end testbed.

Figure 2. Installation Time in Seconds (lower is better)

In terms of day-to-day performance, new Teams produced improvements in application responsiveness across all devices, promising direct time savings for all user types, especially those with a heavy meeting schedule. Application launch (Figure 3) was 3.2x faster on our general purpose testbed, 2.1x faster on our high-end system, and 1.3x faster on our low-end system.

Figure 3. Application Launch Time in Seconds (lower is better)

The time required to join meetings likewise dropped sharply. Our high-end, general purpose, and low-end testbeds proved to be 3.7x, 2.6x, and 1.9x faster, respectively, when joining a meeting from the Teams calendar with new Teams. Figure 4 shows the steep gains in performance.

Figure 4. Seconds Required to Join a Meeting (lower is better)

We saw similar impressive gains in the time required to switch between chats and between channels in new Teams. Taken together, new Teams completed these tasks 1.7x faster than classic Teams, with our low-end machine cutting the time to complete our switch tests on new Teams in half. All test systems showed significant decreases with new Teams that streamline interaction and reduce disruption of thought and conversation (Figure 5).

Figure 5. Combined Time in Seconds to Switch Chats and Channels (lower is better)

Our tests also revealed that new Teams had a significant impact on device subsystems across all three testbeds. Average disk space consumption under new Teams dropped by almost 70% overall (Figure 6), while idle memory consumption—that is, memory consumed while Teams runs in the background—fell by 42% (Figure 7). Our low-end laptop produced the largest gains under new Teams, with idle memory consumption cut by more than half, from 780 MB to 373 MB.

Figure 6. System Disk Capacity Consumption (lower is better)

Figure 7. System Memory Consumption at Idle (lower is better)

We went on to measure memory consumption during four meeting types: one-on-one meetings with screen share and without, and three-on-three meetings with screen share and without. Here we saw an overall 31% reduction in memory consumed with new Teams on our low-end and general purpose laptops, while our high-end machine produced a 20% reduction as seen in Figure 8.

Figure 8. System Memory Consumed During Meetings in MB (lower is better)

Taken together, we saw a 50% average reduction across disk and memory subsystems. These findings are important: More efficient and consistent memory consumption, lower CPU loading, and reduced disk footprint can extend the useful life of aging PCs and improve overall system stability.

Overall, our testing showed significant increases in performance and reduced consumption of device subsystems across all three device tiers. As shown in Figure 9, the low-end system experienced the greatest gains in both areas, illustrating how new Teams is optimized to maximize constrained resources.

Figure 9. Overall Percent Improvement in Resource Consumption and Timed Performance (higher is better)

Organizational Impacts

What do all these results mean for a company? We used the results of our testing to calculate aggregate time savings and cost for different types of employees based on typical activities and usage patterns. For this exercise, we defined two broad user types: line employees representing the bulk of the information workforce and leader employees representing executive and upper management. Finally, we defined three types of organizations to show relative impact: a small business with 250 employees, a midsize business with 2,000 employees, and a large enterprise with 55,000 employees.

As shown in Table 1, the incremental time gained using new Teams adds up. Over the course of a year, a line employee can expect to gain nearly two-and-a-half hours of productivity with new Teams, while management gains four-and-a-half hours. For a small business with 250 employees, that translates into 133 total days of found productivity per year. For a midsize enterprise, the gain is 1,213 days, while a large enterprise realizes 37,776 days of productivity.

Table 1: Annual Time Savings Using New Teams

Extending our analysis to address fiscal impact, we see significant aggregate effects across the organization. Notably, the cost saved per employee increases as the size of the organization goes up—a large enterprise can save $131 per employee versus $78 for a small business. Overall annual savings are also impressive, starting at $20,000 for an SMB to $208,000 for a midsize business to $7.2 million for a large enterprise (Table 2).

Table 2. Annual Cost Dollar Savings Using New Teams

Beyond productivity impacts, the reduced disk and memory load produced by new Teams can reduce help desk activity and extend useful system life across the organization. It can help reduce pressure on the IT supply chain and give management more freedom and flexibility in prioritizing the schedule of system refreshes.

Note that we did not consider the cost of performing the upgrade to new Teams, as the method of carrying this out will vary based on how the IT organization operates.

All results can be found on GigaOm’s GitHub account and are available upon request.

7. Conclusion

Microsoft has made strides to improve the capabilities of Teams over the last few years, reflecting the broad urgency around the uptake of video conferencing and collaboration software during the pandemic and its aftermath. During the period, Microsoft produced a monthly cadence of Teams updates to address emerging needs. The release of new Teams provides a platform for improvement going forward, with a focus on responsiveness and efficient resource consumption that will ease the burden on both end users and IT alike.

As noted earlier, the preview release provides a subset of the features available in classic Teams. Third-party apps, Line of Business (LOB) apps, and advanced calling features such as call queues are not supported at the time of this testing. Additional features will be available in future releases.

This benchmark report addresses the question: How well did Microsoft address the needs of end users and IT organizations with this update of Teams?

Our findings: quite well. Our performance-focused tests produced on average 2x improvement in response times, and we saw a 50% overall decrease in disk and memory resource consumption, representing nearly a 2x improvement in performance—and we saw these results across all three tested device tiers. Some of the results were remarkable. Join time performance more than tripled for our high-end system, while general purpose and low-end systems saw 2.5x and 2x improvements, respectively. We saw similar 2x to 3x improvements in application launch times for our two higher-end platforms.

While the time saved on common activities like switching channels and joining meetings are small, they add up quickly when you are in 200 meetings a month (or more for executives) and multiply that by every person in the organization.

On the system resource side, the measured reduction in disk space is a boon for workstations with smaller disks, like those found in education. And the reduction in memory utilization helps extend the useful life of PCs that might otherwise require prompt replacement. These reductions likewise can extend laptop battery life, again boosting productivity and easing pressure to replace aging systems.

Ultimately, the greatest impact is on the experience of the end user—meetings start faster, join quicker, and the Teams application launches in less time. These are all valuable when fielding back-to-back-to-back meetings with the clock ticking. Repeated over and over throughout the week, the seconds saved really count and enable employees to think and interact more effectively. An upgrade to new Teams will increase productivity and potentially extend the need to upgrade devices.

8. About William McKnight

William McKnight is a former Fortune 50 technology executive and database engineer. An Ernst & Young Entrepreneur of the Year finalist and frequent best practices judge, he helps enterprise clients with action plans, architectures, strategies, and technology tools to manage information.

Currently, William is an analyst for GigaOm Research who takes corporate information and turns it into a bottom-line-enhancing asset. He has worked with Dong Energy, France Telecom, Pfizer, Samba Bank, ScotiaBank, Teva Pharmaceuticals, and Verizon, among many others. William focuses on delivering business value and solving business problems utilizing proven approaches in information management.

9. About Dana Hernandez

Dana Hernandez is a dynamic, accomplished technology leader focused on the application of technology to business strategy and function. Over the last three decades, she had extensive experience with design and implementation of IT solutions in the areas of Finance, Sales, Marketing, Social Platforms, Revenue Management, Accounting, and all aspects of Airline Cargo, including Warehouse Operations. Most recently, she spearheaded technical teams responsible for implementing and supporting all applications for Global Sales for a major airline, owning the technical and business relationship to help drive strategy to meet business needs.

She has led numerous large, complex transformation efforts, including key system merger efforts consolidating companies onto one platform to benefit both companies, and she’s modernized multiple systems onto large ERP platforms to reduce costs, enhance sustainability, and provide more modern functionality to end users.

Throughout her career, Dana leveraged strong analytical and planning skills, combined with the ability to influence others with the common goal of meeting organizational and business objectives. She focused on being a leader in vendor relationships, contract negotiation and management, and resource optimization.

She is also a champion of agile, leading agile transformation efforts across many diverse organizations. This includes heading up major organizational transformations to product taxonomy to better align business with enterprise technology. She is energized by driving organizational culture shifts that include adopting new mindsets and delivery methodologies.

10. About GigaOm

GigaOm provides technical, operational, and business advice for IT’s strategic digital enterprise and business initiatives. Enterprise business leaders, CIOs, and technology organizations partner with GigaOm for practical, actionable, strategic, and visionary advice for modernizing and transforming their business. GigaOm’s advice empowers enterprises to successfully compete in an increasingly complicated business atmosphere that requires a solid understanding of constantly changing customer demands.

GigaOm works directly with enterprises both inside and outside of the IT organization to apply proven research and methodologies designed to avoid pitfalls and roadblocks while balancing risk and innovation. Research methodologies include but are not limited to adoption and benchmarking surveys, use cases, interviews, ROI/TCO, market landscapes, strategic trends, and technical benchmarks. Our analysts possess 20+ years of experience advising a spectrum of clients from early adopters to mainstream enterprises.

GigaOm’s perspective is that of the unbiased enterprise practitioner. Through this perspective, GigaOm connects with engaged and loyal subscribers on a deep and meaningful level.

11. Copyright

© Knowingly, Inc. 2023 "New Microsoft Teams Performance Benchmark" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact sales@gigaom.com.