[11:05:35] I keep being logged out on Commons, literally at every page I load [11:06:15] Three times in a row for now, let's see if I manage to complete my edit. :) [11:06:56] WORKSFORME [11:08:07] On an unrelated note: I miss our quips :/ [11:11:39] hoo|busy: they got imported to https://tools.wmflabs.org/bash/random [11:12:32] hoo|busy: try visiting Special:Preferences or action=edit or action=submit? [11:12:53] hoo|busy: do you mean the GOOD FOR YOU one? [11:13:04] Nemo_bis: Yes and WORKSFORME still [11:13:32] whateverz [11:13:48] login is total lottery for me in the last few years [11:14:37] Extension:AntiBoreout keeping things random :P [14:41:05] This bug is back, and it is really annoying https://phabricator.wikimedia.org/T125793 [14:41:15] Now it is no-js on nowiki [14:42:56] Strange thing is that my laptop running win10 work, while win10 in VirtualBox on my desktop breaks [16:42:16] yurik: hey, so you know, I finally replied to you on the graph extension talk page (from the post from months ago) [16:42:51] djr013, context please? my memory is getting frail :) [16:43:00] https://www.mediawiki.org/wiki/Extension_talk:Graph#Seeking_help_moving_away_from_Wikipedia_template:Line_chart [16:45:59] djr013, ah yes, let me reread it - are you around for the next few min? [16:47:00] yup, almost always on irc :P [16:48:10] plus this project has been bothering me for months, I'm anxious to divest myself of the responsibility of carrying this folder full of spreadsheets on my machine [16:49:07] :o [16:49:31] surely there are archival methods? [16:50:58] I'm sure there are places I could upload them, either as .ods or .csv, but collaborative options that allow them to be updated by anyone every year are few and far between [16:51:30] I think the graph extension can take csv data though, if it's in a place the wiki (in this case wikipedia) is configured to read from? [16:54:22] of course making it easy for other editors to look at and know how to update the data would be a huge plus [16:56:10] though it doesn't help that the source data has occasional obvious clerical errors that have to be manually fixed [17:09:06] something with token/login was changed in the last weeks? [17:09:13] djr013, ok, back :) [17:09:51] djr013, first - do you think it would be possible to use the Graph:Chart for those graphs? [17:10:08] Steinsplitter: yes. why asking? [17:10:13] Steinsplitter, yes. https://lists.wikimedia.org/pipermail/mediawiki-api-announce/2016-January/000102.html [17:10:22] thanks [17:11:24] thanks, i have the +\ problem. need to find out how to fix [17:11:25] yurik: well, it certainly seems capable of it; the difficult part is getting the formatting working and finding a good place and format to put the source data [17:12:00] oh [17:12:10] djr013, does the formatting have to be identical? At the end of the day, I could draw almost anything with the graph ext, BUT, it would mean a separate graph for this [17:12:13] sorry, you mean that template, not the extension? [17:12:19] right [17:12:58] and it would take some time (even though by now i feel pretty comfortable with graphs on the user level, they are still pain at times) [17:13:15] MatmaRex: i am a bit confused with the +\ , where i get the token now? cfrs has the same format? [17:13:18] djr013, and just FYI, this is coming next - http://en.wikipedia.beta.wmflabs.org/wiki/Sparql ;) [17:13:36] data from wikidata! [17:13:49] heh [17:13:57] that'll be very great, and very terrible :P [17:14:12] Steinsplitter: you can get it the same way as before. but you have to make sure to URL-encode it in your requests (previously forgetting to URL-encode did not cause login failures) [17:14:33] because the data is in such a perfect state there :) [17:15:09] yeah having to pay close attention to all that data, and then string up the right queries in the right way [17:15:27] json is bad enough >_> [17:15:36] MatmaRex: oh, thanks! :) [17:15:55] djr013, so anyway, do you think we need to recreate the original look and feel of the graph to the dot? [17:16:51] I'm not sure, but I lean towards saying yes. The benefit of the current template is that it is very compact yet (all things considered) very clear [17:17:40] also as another benefit, the source is in some strange way more human-readable than the current examples given by Graph:Chart [17:17:49] djr013, the biggest problem is mostly with all the minor lines all around the graphs, etc [17:18:56] you mean the scale markers? [17:19:12] as for the source - I agree that it is more human readable than when generated. The benefit of generation is that it does all the data preprocessing, whereas with a you would have to either put that data somewhere else, or have another way of injecting the data [17:19:15] djr013, yes [17:20:17] I tried using the VisualEditor, it just puts all the json directly in article source :P [17:21:55] https://test.wikipedia.org/wiki/Inserting_graph_into_article_test [17:23:11] https://test.wikipedia.org/w/index.php?title=Inserting_graph_into_article_test&action=edit [17:24:04] MatmaRex: sorry for asking again, was there a change at 11 February 2016 as well? [17:25:21] Steinsplitter: that's when wmf.13 was deployed to Wikipedias, so possibly, but i don't quite follow these developments (my bot hasn't broken yet ;) ) [17:25:21] djr013, VE graph is badly broken - 1) it still produces vega1, 2) it requires considerable dev efforts in order to change or add the simplest graph [17:25:32] i think this was a very bad way to go [17:25:46] yeah I realized that pretty quickly lol [17:25:59] so it removes the best thing about wiki - ability by anyone to contribute [17:26:19] just trying to get it to interpret years as years would take considerable research [17:26:32] (for me at least) [17:28:23] you ever used the line chart template? [17:28:54] almost everything in it needs to be configured manually, and re-configured with every new data [17:32:03] djr013, yeah, that's a pain. Biggest question though - we could use some script to update all the pages that call that template, or we could try to fix it to produce the desired graphs internally. The second approach might be better [17:32:23] but that locks us into continuing to use that template's parameter structure [17:32:40] external data is something i have been trying to solve for a while [17:32:43] at least in my head [17:32:58] hmm, I'm not sure I'm reading the two scenarios right [17:33:33] djr013, basically should we change all the wiki articles to use a different template with a different parameter structure, or adjust the existing template and use the underneath [17:33:47] ah [17:34:18] if you updated the existing template you could /maybe/ eliminate a fair number of its manual parameters [17:34:57] https://en.wikipedia.org/w/index.php?title=AFL%E2%80%93CIO&action=edit§ion=1 [17:35:29] huh [17:36:08] djr013, its more than just eliminate, it could be a CSV text passed as a param :) [17:36:17] one thing that could be useful would be to give it different presentation styles; eg floated, centered, basically taken from image presentation modes [17:36:57] before I took the template and shoehorned it into a fake image frame, most of the uses were for large graphs [17:37:09] https://en.wikipedia.org/w/index.php?title=Special:WhatLinksHere&target=Template%3ALine+chart&namespace=0&limit=250 [17:37:52] djr013, wow, i was just thinking about that - tag should support these modes. [17:38:13] we have the same requirements for the maps [17:38:24] ah, yeah [17:41:13] djr013, ok, how about this - I will write vega graph to show data just as you have it. Than we could invoke it like this: {{Graph:WhateverName | {{#GraphTools:CsvToJson | CSV data }} | some additional params}} [17:42:22] well, that is a very simple question for a complex situation, but yes that sounds good :) [17:43:52] keep in mind that though if you simply take the current data, most of the graphs are rounded and truncated because the current template puts anything 1 million or higher into scientific notation [17:45:19] also the source data has occasional issues including missing years (or once in a great while, simply missing data for one of its points), so hopefully the resulting graph gracefully handles this missing data [17:45:43] djr013, how many of these graphs are there? [17:46:23] I forgot the exact count, but pretty most of this list: https://en.wikipedia.org/w/index.php?title=Special:WhatLinksHere&target=Template%3ALine+chart&namespace=0&limit=250 [17:46:52] and more to come, eventually [17:47:38] if there's some way to intersect uses of that template and [[Template:Cite OLMS]] then that's probably it [17:48:41] https://en.wikipedia.org/w/index.php?title=Special:WhatLinksHere/Template:Cite_OLMS&limit=100 [17:48:49] djr013, so just to clarify - its a non-stacked line graph, with x axis being a timeline? [17:49:11] (year) [17:49:19] and y being an arbitrary high value? [17:49:27] yes [17:49:31] multiple lines [17:49:45] do you want to configure line colors from params? [17:49:52] (the membership one could be area, not sure if that is more suitable, but why complicate it more) [17:50:08] djr013, its not a complication, its a different type of graph [17:50:24] graph chart tries to solve all these internally [17:50:30] I think that's more of a style question [17:50:38] re area graph vs line graph [17:50:48] true, but i'm talking about the design of it [17:50:53] i mean development [17:50:59] re line colors, I'm pretty sure I just went with the current template default [17:51:37] right, but with the , you can either supply your own, or let it choose. The big problem is that right now there is no way to make the graph clickable [17:51:37] they probably wouldn't even need to match, at least if the resulting graph has its own legend [17:51:40] no links [17:52:06] graphs could draw its own legend without a problem [17:52:59] I mean really, someone could probably easily convince me to just drop the finances graphs anyway. Interpreting them is pretty arbitrary in most cases. [17:53:40] well, they are probably a good thing, just need to clean up how they are drawn [17:54:51] i keep thinking that i really shouldn't do this, and instead try to implement timeline's tag as a graph extension [17:54:55] but that might be much harder [17:55:02] than to switch all the usages to the new graph [17:55:08] which one is that? [17:55:48] the timeline extension is what draws these old graphs [17:55:53] ah [17:55:56] and it really should be phased out [17:56:51] ok, let me think about it a bit more, keep poking me here. I think we really ought to solve it, but i would hate to spend a lot of effort on the smaller problem when i could solve it for everyone at once with almost as much effort [17:57:16] btw if you want to see something fun [17:57:19] https://en.wikipedia.org/w/index.php?title=Actors%27_Equity_Association&oldid=702683115 [17:57:35] ah yes, i saw that one [17:57:42] :) [17:57:42] not good :) [17:58:02] although finances are so much less boring when accompanied by a laser light show [17:59:18] djr013, have you seen some of the interactive demo graphs? I'm sure we could do something like that ;) [17:59:25] https://www.mediawiki.org/wiki/Extension:Graph/Demo [17:59:36] yurik: I don't mean to be presumptive about it but I think my usecase might be the most finicky at least when it comes to the line chart template [17:59:50] djr013, in what way? [18:00:20] because most of these are just easily formatted large graphs, rather than small embedded thumbnail sized graphs that depend on alignment [18:01:42] yeah I've seen the interactive graphs, they're pretty great :) [18:02:31] djr013, there is a financial example as well :) [18:04:24] oh, the population one could possibly be useful too, but maybe not with the relatively short time period I have source data for [18:04:33] well with one exception [18:04:44] there is an anomaly around 2005 in most of these graphs [18:05:06] where their memberhips suddenly drop by some 10-40% [18:06:01] probably a change in reporting requirements (eg to not include those merely paying agency fees) [18:07:06] [or rather, not to not include them, but rather they then needed to report them as a distinct category, one which is not added into the total membership figure] [18:08:24] https://en.wikipedia.org/wiki/Service_Employees_International_Union#Composition [18:12:25] djr013, is it possible for you to produce all the data in one big CSV? [18:12:56] per article or for the whole entire collection? [18:13:17] the former would be trivial [18:14:28] djr013, later [18:14:43] huh, maybe [18:15:05] is that open data? [18:15:05] I guess I could add another column for union or file-number [18:15:31] yeah it's open data, in the sense that you can navigate your browser to it :P [18:16:14] I went through hell trying to figure out how to link individual reports [18:16:36] that was a whole ordeal even apart from the graphs [18:16:45] djr013, link? [18:17:00] what do you mean by linking individual reports [18:17:55] I'll walk you through it: take for example, AFL-CIO https://en.wikipedia.org/wiki/AFL%E2%80%93CIO#Membership [18:18:07] their file number is 000-106 [18:18:13] https://olms.dol-esa.gov/query/getOrgQry.do [18:18:35] you can enter the number, and from that point on, links will not work, because the forms are purely interactive [18:19:11] (click the organization name for the most useful table) [18:20:52] by some stroke of luck and google-fu I managed to find one or two static directories for individual reports [18:21:21] one link format works for PDF reports scanned from paper copies, and another works for html reports submitted digitally [18:21:26] ridiculous :( [18:21:30] seriously bad [18:21:41] that's the story of how Template:Cite OLMS came to be [18:21:44] the site times out, has no api, ... [18:21:44] https://en.wikipedia.org/wiki/Template:Cite_OLMS [18:22:32] department of labour does everything with labour - in other words - by hand [18:22:52] lots of drones sitting there filling out paperwork [18:22:53] bleh [18:23:31] hmm, I don't think so, with digital reports I think they just look it over and then if they approve it the data goes live [18:23:52] yes, but they have no api - its all very badly done :( [18:24:01] yes very :P [18:24:11] an api would allow us to simply have a permanent URL [18:24:19] and i cannot go to FAQ [18:24:22] site is down [18:25:16] the faq "works" for me but is 404 [18:25:26] "Oh, no! We can't locate the content you requested." [18:25:30] :) [18:25:48] if you mean this one http://webapps.dol.gov/dolfaq/ [18:27:46] also sadly if I want data from before 2000 I'd need to request paper copies and pay for it [18:27:47] lol [18:28:46] they'd probably have to send some courageous spelunker into some salt mine archives to fetch it all [18:30:52] better than the UK certification office, which IIRC only has reports for the last two/three years [18:31:05] probably whenever they designed the current iteration of their websire [18:32:15] but yes perhaps it becomes apparent why I got exhausted gathering and formatting all the data :) [18:43:59] yurik: do you need me to produce that aggregate csv? [18:44:19] djr013, i'm thinking if that would be good or not [18:44:27] ah [18:44:53] djr013, if you need all that data to draw a graph, that means we need to copy it. In which case, we might as well copy all of it [18:45:02] there is no point in copying it one piece at a time [18:45:05] error prone [19:57:52] someone was realizing the DOL is a little obnoxious