Liwen's Notes

Recent Notes

Manual - WebPageTest

Feb 24, 20235 min read

Use WebPageTest APIs

To use WebPageTest API, you’ll need to obtain an API key first.

Request an API Key 🔑 (link)

-----BEGIN PGP MESSAGE-----

hQIOAxdwc3QWykEfEAgAsqXDoRDvJ+gxMMwmKMJZHgo+sdR30/4sR4l4BT/2BDa3
5BgVWowfMFEgNCzp6YQ2rfAFNtJ8NxqcedIK7b7q+aQQ1HuD3WwIHEil/QCHwmK1
vTLJ+gWEL/XwPZmUfTssg19OkOzj7BtSutG7vrOm5fNdN0qGbP7kmsSIFHFyw7GY
/c6CC6jEHIM573dtvHZdKHPudZUc9V96s2rJQkYf8vbHTXAbSGH5WXuKY13EnfZb
z72hO224JcppvlGRbfuIs2KmTpit9TRF3brT2S6iRxJfgsa8VxQsM6c4OLhjNRkJ
TILKnSU+fn+h7Tgvw3y7JJRHNYNY2IYoeh9KRtL9lQf+KDKWNLx9R1QVHwqh3bVu
2AyK2T1IU2WNJi+yG+Ar77aTwNGdAFrcyNVY2ec0TiP5AE1WomJ39+Lr3yhavl58
jprBoJUHpQLWFRYU/p3VNFizVBMU+q+kWnq32ge4EgYtH6p3TCop53U8zlz7ZkM2
Ix6q82WXy3rsCddUnDEx3U5C1eXo1YiHxrdw/raGRVuFQmgpEqkOIcuwWeD8eImm
k4/LzPIQgLoBqZLRNP7cwbe+q9Jyc7V/ypWh2kYyDRxgFBEXgYNDabn0JDCPbcZ1
ufYwYnzx9+1Dxr9aZo+Bihsyn9Rn9mUAIyDH6agwJuVYg1/IMhXJQt5EIW8TFIKG
ydJRAeY+Hf66Ba0IZNhabh+RwPt2kEt/EDU7YMjcIpxhkyJLRFHc//ldonBTExfS
n5AR0+/qDttsne6k5ldo77/AXSgR7LS7vxrNWB9B4/Dn8Zuh
=KR1T
-----END PGP MESSAGE-----


Your API key is: 
 
The API key is limited to 200 page loads per day. Each run, first or repeat view
counts as a page load (10 runs, first and repeat view would be 20 page
loads). If you need to do more testing than that allows then you should consider
running a private instance:
https://sites.google.com/a/webpagetest.org/docs/private-instances
 
The list of current locations available for API testing here:
https://www.webpagetest.org/getLocations.php?f=html&k=A.
 
The following browser/location combinations are available:
 
Dulles_IE11
Dulles_Edge
Dulles:Chrome
Dulles:Canary
Dulles:Firefox
Dulles:Firefox Nightly
Dulles_MotoG:Moto G - Chrome
Dulles_MotoG:Moto G - Chrome Canary
Dulles_MotoG:Moto G - Chrome Beta
Dulles_MotoG:Moto G - Chrome Dev
Dulles_MotoG:Moto G - UC Browser
Dulles_MotoG:Moto G - UC Mini
Dulles_MotoG:Moto G - Opera Mini (Extreme)
Dulles_MotoG:Moto G - Opera Mini (High)
Dulles_MotoG:Moto G - Firefox
Dulles_MotoG4:Moto G4 - Chrome
Dulles_MotoG4:Moto G4 - Chrome Canary
Dulles_MotoG4:Moto G4 - Chrome Beta
Dulles_MotoG4:Moto G4 - Chrome Dev
Dulles_MotoG4:Moto G4 - Samsung Internet
Dulles_MotoG4:Moto G4 - UC Browser
Dulles_MotoG4:Moto G4 - UC Mini
Dulles_MotoG4:Moto G4 - Opera Mini (Extreme)
Dulles_MotoG4:Moto G4 - Opera Mini (High)
Dulles_MotoG4:Moto G4 - Firefox
Dulles_Thinkpad:Chrome
Dulles_Thinkpad:Canary
Dulles_Thinkpad:Firefox
Dulles_Thinkpad:Firefox Beta
Dulles_Thinkpad:Firefox Nightly
Dulles_Thinkpad:Microsoft Edge
Dulles_Thinkpad:IE 11
London_EC2:Chrome
London_EC2:Firefox
ec2-us-east-1:Chrome
ec2-us-east-1:Chrome Beta
ec2-us-east-1:Chrome Canary
ec2-us-east-1:Firefox
ec2-us-east-1:Firefox Nightly
ec2-us-east-1:Opera
ec2-us-east-1:Opera Beta
ec2-us-east-1:Opera Developer
ec2-us-west-1:Chrome
ec2-us-west-1:Chrome Beta
ec2-us-west-1:Chrome Canary
ec2-us-west-1:Firefox
ec2-us-west-1:Firefox Nightly
ec2-us-west-1:Opera
ec2-us-west-1:Opera Beta
ec2-us-west-1:Opera Developer
ec2-eu-west-1:Chrome
ec2-eu-west-1:Chrome Beta
ec2-eu-west-1:Chrome Canary
ec2-eu-west-1:Firefox
ec2-eu-west-1:Firefox Nightly
ec2-eu-west-1:Opera
ec2-eu-west-1:Opera Beta
ec2-eu-west-1:Opera Developer
ec2-eu-central-1:Chrome
ec2-eu-central-1:Chrome Beta
ec2-eu-central-1:Chrome Canary
ec2-eu-central-1:Firefox
ec2-eu-central-1:Firefox Nightly
ec2-eu-central-1:Opera
ec2-eu-central-1:Opera Beta
ec2-eu-central-1:Opera Developer
ec2-ap-northeast-2:Chrome
ec2-ap-northeast-2:Chrome Beta
ec2-ap-northeast-2:Chrome Canary
ec2-ap-northeast-2:Firefox
ec2-ap-northeast-2:Firefox Nightly
ec2-ap-northeast-2:Opera
ec2-ap-northeast-2:Opera Beta
ec2-ap-northeast-2:Opera Developer
ec2-ap-northeast-1:Chrome
ec2-ap-northeast-1:Chrome Beta
ec2-ap-northeast-1:Chrome Canary
ec2-ap-northeast-1:Firefox
ec2-ap-northeast-1:Firefox Nightly
ec2-ap-northeast-1:Opera
ec2-ap-northeast-1:Opera Beta
ec2-ap-northeast-1:Opera Developer
ap-south-1:Chrome
ap-south-1:Chrome Beta
ap-south-1:Chrome Canary
ap-south-1:Firefox
ap-south-1:Firefox Nightly
ap-south-1:Opera
ap-south-1:Opera Beta
ap-south-1:Opera Developer
ec2-ap-southeast-1:Chrome
ec2-ap-southeast-1:Chrome Beta
ec2-ap-southeast-1:Chrome Canary
ec2-ap-southeast-1:Firefox
ec2-ap-southeast-1:Firefox Nightly
ec2-ap-southeast-1:Opera
ec2-ap-southeast-1:Opera Beta
ec2-ap-southeast-1:Opera Developer
ec2-ap-southeast-2:Chrome
ec2-ap-southeast-2:Chrome Beta
ec2-ap-southeast-2:Chrome Canary
ec2-ap-southeast-2:Firefox
ec2-ap-southeast-2:Firefox Nightly
ec2-ap-southeast-2:Opera
ec2-ap-southeast-2:Opera Beta
ec2-ap-southeast-2:Opera Developer
ec2-sa-east-1:Chrome
ec2-sa-east-1:Chrome Beta
ec2-sa-east-1:Chrome Canary
ec2-sa-east-1:Firefox
ec2-sa-east-1:Firefox Nightly
ec2-sa-east-1:Opera
ec2-sa-east-1:Opera Beta
ec2-sa-east-1:Opera Developer

Scripting

WebpageTest supports scripting.

logData    0
 
// put any urls you want to navigate
navigate    https://www.thesun.co.uk
navigate    https://www.thesun.co.uk/travel
 
logData    1
 
// this step will get recorded
navigate    https://www.thesun.co.uk/travel/8572000

Run WebPageTest locally with Docker

#  Pull the WebPageTest image
docker pull webpagetest/server
docker pull webpagetest/agent

Run the server:

# -d runs the container in "detached" mode in the background
# -p specifies the port to expose.
docker run -d -p 4000:80 webpagetest/server

Run the agent:

# --network="host" allows the agent to communicate with the server container
# -e specifies an environment variable. WPT requires `SERVER_URL` and `LOCATION` be defined.
docker run -d -p 4001:80 \
  --network="host" \
  -e "SERVER_URL=http://localhost:4000/work/" \
  -e "LOCATION=Test" \
  webpagetest/agent

Start it up:

Head over to http://localhost:4000 . You should see the frontend for WebPagetest. There’s a built in configuration check if you hit http://localhost:4000/install

Traffic shaping on Mac OS is officially supported, however if you are using Docker it will not work as Docker runs in a linux headless agent.)

Traffic Shaping MacOS with Docker Workaround:

You can remove traffic shaping by doing two things :

  • Setting a dummy value for connectivity in your settings/locations.ini{.verbatim} file on your server
  • And setting --shaper none{.verbatim} when the agent is starting up

Let’s create a new Docker image based on the original WPT agent/server images but with modifications to the configuration.

Server:

Setup a new folder that has two files, a Dockerfile{.verbatim} and a locations.ini{.verbatim}

./Dockerfile{.verbatim}

FROM webpagetest/server
ADD locations.ini /var/www/html/settings/

./locations.ini{.verbatim}

[locations]
1=Test_loc
[Test_loc]
1=Test
label=Test Location
group=Desktop
[Test]
browser=Chrome,Firefox
label="Test Location"
connectivity=LAN

This defines a new docker image that pulls from webpagetest/server, replacing the locations.ini file with one that will work for OS X, particularly setting connectivity to remove traffic shaping.

# -t defines the name we are giving to this image
docker build -t local-wptserver .

Agent:

Setup a new folder that has two files, a Dockerfile{.verbatim} and a script.sh{.verbatim}

./Dockerfile{.verbatim}

FROM webpagetest/agent
ADD script.sh /
ENTRYPOINT /script.sh

./locations.ini{.verbatim}

#!/bin/bash
set -e
if [ -z "$SERVER_URL" ]; then
  echo >&2 'SERVER_URL not set'
  exit 1
fi
if [ -z "$LOCATION" ]; then
  echo >&2 'LOCATION not set'
  exit 1
fi
EXTRA_ARGS=""
if [ -n "$NAME" ]; then
  EXTRA_ARGS="$EXTRA_ARGS --name $NAME"
fi
 
python /wptagent/wptagent.py --server $SERVER_URL --location $LOCATION $EXTRA_ARGS --xvfb --dockerized -vvvvv --shaper none

Make sure script.sh is executable:

chmod u+x script.sh

And now we can build our agent image:

# -t defines the name we are giving to this image
docker build -t local-wptagent .

Start it up:

Stop the running WPT containers:

docker ps
# CONTAINER ID        IMAGE
# 5e2374829136        webpagetest/agent
# 1cf57d494fc8        webpagetest/server
docker stop 5e2374829136
docker stop 1cf57d494fc8

Start up your local WPT containers:

docker run -d -p 4000:80 local-wptserver
docker run -d -p 4001:80 \
  --network="host" \
  -e "SERVER_URL=http://localhost:4000/work/" \
  -e "LOCATION=Test" \
  local-wptagent

You should now be able hit http://localhost:4000 and run webpage tests against your local environments.

Graph View

Backlinks

  • No backlinks found