Changes to bp_has_activities() queries in BP 2.1

Activity queries were largely overhauled in BuddyPress 2.0, leading to major performance improvements. These improvements were focused on the paginated query, the SELECT that contained a LIMIT clause based on the per_page value. The remaining bottleneck for activity pages on large installations was the other SELECT COUNT(*) query performed in BP_Activity_Activity::get() to get the total number of items matching the query params, a query that is much less amenable to optimization. The kicker is that the value returned by this query is almost never used in BuddyPress; the most common use for the total count figure is to generate pagination HTML, but by default, BuddyPress does not show true pagination for activity anywhere on the front end – just a generic “Load More” button at the bottom of the stream.

BuddyPress 2.1 will include changes to address this bottleneck. (See #5629 and r8491.) The SELECT COUNT(*) query will no longer take place by default, and the logic for displaying the “Load More” button has been altered to work without this query. Most sites will not notice any difference (except for the increase in speed!). However, there are a few cases where developers may need to make modifications to preserve some functionality:

  • noscript support – The “Load More” button on activity page uses Javascript. To cover browsers with JS disabled, a <noscript> block appeared in the activity-loop.php template that contained traditional pagination markup. This pagination will no longer generate properly as of BP 2.1. In BP’s bp-legacy templates, we have removed the <noscript> block and altered the “Load More” href to be populated by bp_activity_load_more_link(). However, if your theme is a bp-default derivative, or if you have overridden activity-loop.php in your theme, you will need to make the mods yourself to support non-JS browsers. Either reproduce the core change, or add count_total=count_query to the bp_has_activities() arguments.
  • Custom activity queries – If you have built a plugin or another customization that uses BP’s activity query functions (bp_has_activities(), bp_activity_get(), BP_Activity_Activity::get()), and if your customization relies on the 'total' or 'total_activity_count' value, you’ll need to make the necessary modifications to work properly with BP 2.1. In most cases, this just means passing 'count_total' => 'count_query' as one of the function args.

Apologies to those for whom this will cause an inconvenience – normally we try not to break backward compatibility for any reason, but it was impossible to ensure that all installations would receive the performance benefits without affecting a certain subset of installations (we did try to minimize impact).

Questions? Please ask in the comments below.

#activity, #bp_activity_activity, #bp_activity_get, #bp_has_activities

BP 2.0.1 is out: http://buddypress.org/2014/05/buddypress-2-0-1/

BP 2.0.1 is out: http://buddypress.org/2014/05/buddypress-2-0-1/

BuddyPress 2.0 “Juliana” has been…

BuddyPress 2.0 “Juliana” has been released: http://buddypress.org/2014/04/buddypress-2-0-juliana/

Special thanks to the readers of this blog who have helped with development and testing during this dev period. Go enjoy a slice of pizza!

BP 2.0 RC 1: http://buddypress.org/2014/04/buddypress-2-0-release-candidate/

BP 2.0 RC 1: http://buddypress.org/2014/04/buddypress-2-0-release-candidate/

BP 2.0-beta2 is now available:…

BP 2.0-beta2 is now available: http://buddypress.org/2014/04/buddypress-2-0-beta2/

One of the primary focuses…

One of the primary focuses of BP 2.0 development has been performance. In this post, I’ll share a few numbers that demonstrate the improvements you can expect in the new version.

For testing, I set up a fresh installation of WP 3.8.1, running Twenty Fourteen and the default widget configuration. Using wp-cli-buddypress, I generated 30,000 members and 60,000 activity items. I toggled between various setups, running the latest BP 1.9.x as well as the latest BP trunk (2.0.x). The tests labeled “APC On” were run using APC with a default config (specifically, APCu on PHP 5.5.10) and the WP APC plugin. In these cases, I also primed the cache with a page refresh. Tests labeled “APC Off” do not have persistent object caching enabled.

What follows are statistics about MySQL (number of queries and the total query time). In many cases, there are corresponding improvements in script execution time and memory footprint, but these latter numbers are so dependent on other environmental features that I’ve decided not to include them here. You should take things like “query time” with a grain of salt; the numbers are useful mainly to visualize relative gains. But it’s worth noting that the improvements in query time will become more significant as sites become larger than the dummy site that these numbers come from.

  1. Baseline
    Here are some numbers with BP deactivated, so you can get a sense of what the default WP installation is doing:

    Home page – BP deactivated – APC Off

    # queries: 30
    query time: 0.005503s


    Home page – BP deactivated – APC On

    # queries: 9
    query time: 0.002269s

  2. Activity
    Several important changes were made to the way that BuddyPress activity is queried, including significant improvements in object caching and splitting the query to avoid costly JOINs. Here are some results of looking at the main Activity directory page, default filters, as a logged-out visitor.

    BP 1.9.2 – APC Off

    # queries: 80
    Query time: 0.111316s


    BP 2.0 – APC Off

    # queries: 59
    Query time: 0.046568s


    BP 1.9.2 – APC On

    # queries: 10
    Query time: 0.002157s


    BP 2.0 – APC On

    # queries: 5
    Query time: 0.001317s

  3. Members
    The efficiency of members queries was another area of intense focus during the 2.0 dev cycle. The way that last_activity data is stored and queried was overhauled, and a large number of improvements were made to userdata caching. The following data is from the 234th page of the Members directory, sorted by Last Active (/members/?upage=234), as seen by a logged-out visitor.

    BP 1.9.2 – APC Off

    # queries: 63
    Query time: 0.197926s


    BP 2.0 – APC Off

    # queries: 39
    Query time: 0.0952112s


    BP 1.9.2 – APC On

    # queries: 17
    Query time: 0.184889s


    BP 2.0 – APC On

    # queries: 10
    Query time: 0.091074s

  4. Logged-in data
    Users who are logged in to a BP site carry a lot of data along with them on every page of the site. Userdata is used to build navigation links, to build the admin bar, to populate the Notifications dropdown, etc. We’ve made an effort to make these queries less redundant and more efficent. Here are some numbers from a home page visit by a logged-in user.

    BP 1.9.2 – APC Off

    # queries: 49
    Query time: 0.010246s


    BP 2.0 – APC Off

    # queries: 45
    Query time: 0.008786s


    BP 1.9.2 – APC On

    # queries: 16
    Query time: 0.0042662


    BP 2.0 – APC On

    # queries: 4
    Query time: 0.0012240

You can see that the improvements are significant. The number of queries per page load in each case has been cut by up to 50%, and query time, on average, has been cut by more than 50%. And remember that these numbers are total values – it’s BP on top of WP’s standard queries. That is to say that the query footprint attributable to BP itself has been reduced by up to 75% in many places.

This is not an exhaustive list of benchmarks, but it’s indicative of improvements you should see across most sections of BP sites after the 2.0 upgrade. The improvements will be especially noticeable on very BP installations. If this isn’t good motivation for you to help with the 2.0 beta, I don’t know what is :)

2.0 Beta 1 is now…

2.0 Beta 1 is now available! http://buddypress.org/2014/03/buddypress-2-0-beta-1-is-now-available/