[Wikimetrics] Report fails

Discussion:

James Hare

2015-03-18 19:47:31 UTC

I am trying to run a report measuring rolling active editor and newly registered for a cohort of about ~20,000 accounts (actually only about ~1,300 Commons usernames, but I opted to have it extended to their global accounts in case they also did stuff outside of Commons). I have now tried to run the report twice and it has failed. Anyone know what is going on?

Thanks,
James

â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc

Nuria Ruiz

2015-03-18 21:02:43 UTC

Permalink

James,

There are probably better metrics for your purposes, could you let us know
what data are you interested on? Once we know that we can take a look to
see if there are any issues with the tool itself.

Thanks,

Nuria

Post by James Hare
I am trying to run a report measuring rolling active editor and newly
registered for a cohort of about ~20,000 accounts (actually only about
~1,300 Commons usernames, but I opted to have it extended to their global
accounts in case they also did stuff outside of Commons). I have now tried
to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

James Hare

2015-03-18 21:04:01 UTC

Permalink

I want to know, for a period from July 1 to September 30, which of the ~1,300 contributors are new, and which are active editors, per the global metrics meanings of those terms. Let me know fi you need a list of usernames from me.

Thanks,
James

â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc

Post by Nuria Ruiz
James,
There are probably better metrics for your purposes, could you let us know what data are you interested on? Once we know that we can take a look to see if there are any issues with the tool itself.
Thanks,
Nuria

Post by James Hare
I am trying to run a report measuring rolling active editor and newly registered for a cohort of about ~20,000 accounts (actually only about ~1,300 Commons usernames, but I opted to have it extended to their global accounts in case they also did stuff outside of Commons). I have now tried to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

Nuria Ruiz

2015-03-18 23:24:51 UTC

Permalink

James,

If you could send the cohort you are using (if it is not private) it will
help when troubleshooting.

Thanks,

Nuria

Post by James Hare
I want to know, for a period from July 1 to September 30, which of the
~1,300 contributors are new, and which are active editors, per the global
metrics meanings of those terms. Let me know fi you need a list of
usernames from me.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
James,
There are probably better metrics for your purposes, could you let us know
what data are you interested on? Once we know that we can take a look to
see if there are any issues with the tool itself.
Thanks,
Nuria
I am trying to run a report measuring rolling active editor and newly
registered for a cohort of about ~20,000 accounts (actually only about
~1,300 Commons usernames, but I opted to have it extended to their global
accounts in case they also did stuff outside of Commons). I have now tried
to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

Dan Andreescu

2015-03-19 16:31:46 UTC

Permalink

James,

I'm cc-ing Jonathan as this issue looks to have two components.

First, James, I ran the Edits metric on your cohort and it worked ok. I am
running the Rolling New Active Editor metric and it seems a bit slow, so
we're looking into that.

Second, how exactly are these reports run? Like, are you using Rolling New
Active Editor and doing it in chunks of 30 days over the period you need?
Could you detail this, as these rolling metrics are very slow and were not
really meant for this kind of use. We can adjust our system accordingly
(maybe we can make faster metrics that get to what you need better).

James Hare

2015-03-19 16:42:44 UTC

Permalink

Rolling new active editor may not be the metric I need. Just, for that period of time, I need to know which of the editors were active editors (5+ edits/month) and which had created their accounts in that period.

Thanks,
James

â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc

Post by Nuria Ruiz
James,
I'm cc-ing Jonathan as this issue looks to have two components.
First, James, I ran the Edits metric on your cohort and it worked ok. I am running the Rolling New Active Editor metric and it seems a bit slow, so we're looking into that.
Second, how exactly are these reports run? Like, are you using Rolling New Active Editor and doing it in chunks of 30 days over the period you need? Could you detail this, as these rolling metrics are very slow and were not really meant for this kind of use. We can adjust our system accordingly (maybe we can make faster metrics that get to what you need better).

Post by James Hare
I am trying to run a report measuring rolling active editor and newly registered for a cohort of about ~20,000 accounts (actually only about ~1,300 Commons usernames, but I opted to have it extended to their global accounts in case they also did stuff outside of Commons). I have now tried to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

Edward Galvez

2015-03-19 17:02:56 UTC

Permalink

Hi James and Kevin,

Here are the global metrics specifications for newly registered and active editors involved for wmf grantees. James, please let us know if you are looking for other metrics or if these are what you need. I ask because the metrics you are specifying sound different.

https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_newly_registered_users

https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_active_editors_involved

Thanks!
Edward

Post by James Hare
Rolling new active editor may not be the metric I need. Just, for that period of time, I need to know which of the editors were active editors (5+ edits/month) and which had created their accounts in that period.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc

Post by James Hare
I am trying to run a report measuring rolling active editor and newly registered for a cohort of about ~20,000 accounts (actually only about ~1,300 Commons usernames, but I opted to have it extended to their global accounts in case they also did stuff outside of Commons). I have now tried to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

James Hare

2015-03-19 18:44:22 UTC

Permalink

(Whoâs Kevin?)

For Rolling Active Editor, I decided to chunk it into month-long periods. So I am trying the first one for 30 days prior to June 1, 2014. Yet that report nonetheless failed.

The report for newly registered worked out just fine.

â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc

Post by Edward Galvez
Hi James and Kevin,
Here are the global metrics specifications for newly registered and active editors involved for wmf grantees. James, please let us know if you are looking for other metrics or if these are what you need. I ask because the metrics you are specifying sound different.
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_newly_registered_users
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_active_editors_involved
Thanks!
Edward

Post by James Hare
I am trying to run a report measuring rolling active editor and newly registered for a cohort of about ~20,000 accounts (actually only about ~1,300 Commons usernames, but I opted to have it extended to their global accounts in case they also did stuff outside of Commons). I have now tried to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

Edward Galvez

2015-03-19 18:47:31 UTC

Permalink

Sorry - I meant Dan!

And thanks for the clarification James (those metrics make sense)

Post by James Hare
(Whoâs Kevin?)
For Rolling Active Editor, I decided to chunk it into month-long periods.
So I am trying the first one for 30 days prior to June 1, 2014. Yet that
report nonetheless failed.
The report for newly registered worked out just fine.
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
Hi James and Kevin,
Here are the global metrics specifications for newly registered and active
editors involved for wmf grantees. James, please let us know if you are
looking for other metrics or if these are what you need. I ask because the
metrics you are specifying sound different.
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_newly_registered_users
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_active_editors_involved
Thanks!
Edward
Rolling new active editor may not be the metric I need. Just, for that
period of time, I need to know which of the editors were active editors (5+
edits/month) and which had created their accounts in that period.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
James,
I'm cc-ing Jonathan as this issue looks to have two components.
First, James, I ran the Edits metric on your cohort and it worked ok. I
am running the Rolling New Active Editor metric and it seems a bit slow, so
we're looking into that.
Second, how exactly are these reports run? Like, are you using Rolling
New Active Editor and doing it in chunks of 30 days over the period you
need? Could you detail this, as these rolling metrics are very slow and
were not really meant for this kind of use. We can adjust our system
accordingly (maybe we can make faster metrics that get to what you need
better).
I want to know, for a period from July 1 to September 30, which of the
~1,300 contributors are new, and which are active editors, per the global
metrics meanings of those terms. Let me know fi you need a list of
usernames from me.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
James,
There are probably better metrics for your purposes, could you let us know
what data are you interested on? Once we know that we can take a look to
see if there are any issues with the tool itself.
Thanks,
Nuria
I am trying to run a report measuring rolling active editor and newly
registered for a cohort of about ~20,000 accounts (actually only about
~1,300 Commons usernames, but I opted to have it extended to their global
accounts in case they also did stuff outside of Commons). I have now tried
to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

--
Edward Galvez
Program Evaluation Associate
Wikimedia Foundation

Kevin Leduc

2015-03-19 18:55:50 UTC

Permalink

I am Kevin (Product Manager for the Analytics team) and I am keeping an eye
on this thread and what we could do better.

Post by Edward Galvez
Sorry - I meant Dan!
And thanks for the clarification James (those metrics make sense)

Post by James Hare
(Whoâs Kevin?)
For Rolling Active Editor, I decided to chunk it into month-long periods.
So I am trying the first one for 30 days prior to June 1, 2014. Yet that
report nonetheless failed.
The report for newly registered worked out just fine.
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
Hi James and Kevin,
Here are the global metrics specifications for newly registered and
active editors involved for wmf grantees. James, please let us know if you
are looking for other metrics or if these are what you need. I ask because
the metrics you are specifying sound different.
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_newly_registered_users
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_active_editors_involved
Thanks!
Edward
Rolling new active editor may not be the metric I need. Just, for that
period of time, I need to know which of the editors were active editors (5+
edits/month) and which had created their accounts in that period.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
James,
I'm cc-ing Jonathan as this issue looks to have two components.
First, James, I ran the Edits metric on your cohort and it worked ok. I
am running the Rolling New Active Editor metric and it seems a bit slow, so
we're looking into that.
Second, how exactly are these reports run? Like, are you using Rolling
New Active Editor and doing it in chunks of 30 days over the period you
need? Could you detail this, as these rolling metrics are very slow and
were not really meant for this kind of use. We can adjust our system
accordingly (maybe we can make faster metrics that get to what you need
better).
I want to know, for a period from July 1 to September 30, which of the
~1,300 contributors are new, and which are active editors, per the global
metrics meanings of those terms. Let me know fi you need a list of
usernames from me.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
James,
There are probably better metrics for your purposes, could you let us
know what data are you interested on? Once we know that we can take a look
to see if there are any issues with the tool itself.
Thanks,
Nuria
I am trying to run a report measuring rolling active editor and newly
registered for a cohort of about ~20,000 accounts (actually only about
~1,300 Commons usernames, but I opted to have it extended to their global
accounts in case they also did stuff outside of Commons). I have now tried
to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

--
Edward Galvez
Program Evaluation Associate
Wikimedia Foundation
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

Dan Andreescu

2015-03-19 19:19:37 UTC

Permalink

I see the potential problem then. James, try something else for me:

Instead of Rolling Active Editor, which is a heavy and scary metric, try
using Edits with:

Start: *June 1st 2014*
End: *October 1st 2014*
Time Series by: *month*
Namespaces: if you leave this blank, it will get all namespaces, which
sounds like what you're looking for

Then you can look at the CSV result and do some quick analysis on which
editors had 5+ edits per month. The "Time Series by" parameter is key
there.

Just so you don't think I'm making you a test subject, I tried this myself
for 2 months of data and the report finished quickly. So I'm confident
that it will work, but let us know if you have trouble.

Also, while running Rolling Active Editor we ran into performance issues
and failures as we predicted above. This is an infrastructure issue that's
very tricky to fix, so there's probably nothing we can do in the short
term. Either way we've opened a ticket and are looking at the problem:
https://phabricator.wikimedia.org/T93217

Post by Kevin Leduc
I am Kevin (Product Manager for the Analytics team) and I am keeping an
eye on this thread and what we could do better.

Post by Edward Galvez
Sorry - I meant Dan!
And thanks for the clarification James (those metrics make sense)

Post by James Hare
(Whoâs Kevin?)
For Rolling Active Editor, I decided to chunk it into month-long
periods. So I am trying the first one for 30 days prior to June 1, 2014.
Yet that report nonetheless failed.
The report for newly registered worked out just fine.
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
Hi James and Kevin,
Here are the global metrics specifications for newly registered and
active editors involved for wmf grantees. James, please let us know if you
are looking for other metrics or if these are what you need. I ask because
the metrics you are specifying sound different.
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_newly_registered_users
https://meta.wikimedia.org/wiki/Grants:Learning_patterns/Number_of_active_editors_involved
Thanks!
Edward
Rolling new active editor may not be the metric I need. Just, for that
period of time, I need to know which of the editors were active editors (5+
edits/month) and which had created their accounts in that period.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
James,
I'm cc-ing Jonathan as this issue looks to have two components.
First, James, I ran the Edits metric on your cohort and it worked ok. I
am running the Rolling New Active Editor metric and it seems a bit slow, so
we're looking into that.
Second, how exactly are these reports run? Like, are you using Rolling
New Active Editor and doing it in chunks of 30 days over the period you
need? Could you detail this, as these rolling metrics are very slow and
were not really meant for this kind of use. We can adjust our system
accordingly (maybe we can make faster metrics that get to what you need
better).
I want to know, for a period from July 1 to September 30, which of the
~1,300 contributors are new, and which are active editors, per the global
metrics meanings of those terms. Let me know fi you need a list of
usernames from me.
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
James,
There are probably better metrics for your purposes, could you let us
know what data are you interested on? Once we know that we can take a look
to see if there are any issues with the tool itself.
Thanks,
Nuria
I am trying to run a report measuring rolling active editor and newly
registered for a cohort of about ~20,000 accounts (actually only about
~1,300 Commons usernames, but I opted to have it extended to their global
accounts in case they also did stuff outside of Commons). I have now tried
to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

--
Edward Galvez
Program Evaluation Associate
Wikimedia Foundation
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

James Hare

2015-03-19 19:48:15 UTC

Permalink

That did the trick. Thank you, Dan!

â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc

Start: June 1st 2014
End: October 1st 2014
Time Series by: month
Namespaces: if you leave this blank, it will get all namespaces, which sounds like what you're looking for
Then you can look at the CSV result and do some quick analysis on which editors had 5+ edits per month. The "Time Series by" parameter is key there.
Just so you don't think I'm making you a test subject, I tried this myself for 2 months of data and the report finished quickly. So I'm confident that it will work, but let us know if you have trouble.
Also, while running Rolling Active Editor we ran into performance issues and failures as we predicted above. This is an infrastructure issue that's very tricky to fix, so there's probably nothing we can do in the short term. Either way we've opened a ticket and are looking at the problem: https://phabricator.wikimedia.org/T93217

I am Kevin (Product Manager for the Analytics team) and I am keeping an eye on this thread and what we could do better.

Post by Edward Galvez
Sorry - I meant Dan!
And thanks for the clarification James (those metrics make sense)

Post by James Hare
I am trying to run a report measuring rolling active editor and newly registered for a cohort of about ~20,000 accounts (actually only about ~1,300 Commons usernames, but I opted to have it extended to their global accounts in case they also did stuff outside of Commons). I have now tried to run the report twice and it has failed. Anyone know what is going on?
Thanks,
James
â
James Hare
President, Wikimedia DC
http://wikimediadc.org
@wikimediadc
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

--
Edward Galvez
Program Evaluation Associate
Wikimedia Foundation
_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics

_______________________________________________
Wikimetrics mailing list
https://lists.wikimedia.org/mailman/listinfo/wikimetrics