[00:01] <wgrant> Now it's time to play everyone's favourite game: "work out how to run that untested Soyuz script that probably hasn't run outside cocoplum for 5 years."
[00:26] <lifeless> wgrant: archive:+index is still slow :(
[00:26] <lifeless> wgrant: 5K > 10 seconds
[00:26] <lifeless> yesterday
[00:31] <wgrant> lifeless: Yeah.
[00:31] <wgrant> lifeless: I can probably get it down slightly by removing the remaining couple of duplicate queries.
[00:31] <wgrant> But the main issue is that the remaining queries are slow.
[00:31] <wgrant> Needs a stub.
[01:18] <lifeless> wgrant: if he starts while you're still working today, you could ask him about it
[01:18] <lifeless> I think I pointed him at it already, he may have some ideas
[01:28] <wgrant> lifeless: I will hopefully not be around much this evening.
[01:28] <wgrant> But I will poke him if our hours intersect.
[01:59] <LPCIBot> Project db-devel build (252): STILL FAILING in 3 hr 32 min: https://hudson.wedontsleep.org/job/db-devel/252/
[02:09] <lifeless> [02:09] <lifeless>     Hard / Soft  Page ID
[02:09] <lifeless>       33 /  189  BugTask:+index
[02:09] <lifeless>       15 /  358  Distribution:+bugs
[02:09] <lifeless>       14 /    2  Archive:+copy-packages
[02:09] <lifeless>        9 /    0  Bug:EntryResource:markUserAffected
[02:09] <lifeless>        8 / 3280  Archive:+index
[02:09] <lifeless>        8 /    3  ProjectGroup:+milestones
[02:09] <lifeless>        5 /  392  Distribution:+bugtarget-portlet-bugfilters-stats
[02:09] <lifeless>        5 /    8  NullBugTask:+index
[02:09] <lifeless>        4 /   92  ProjectGroupSet:CollectionResource:#project_groups
[02:09] <lifeless>        4 /   62  MailingListApplication:MailingListAPIView
[02:27] <LPCIBot> Project devel build (338): STILL FAILING in 3 hr 12 min: https://hudson.wedontsleep.org/job/devel/338/
[02:27] <LPCIBot> Launchpad Patch Queue Manager: [r=sinzui][ui=none][bug=579502] Do not send bug email to distro owner
[02:27] <LPCIBot> if the distro does not use Lauchpad for bugs.
[05:10] <LPCIBot> Project db-devel build (253): STILL FAILING in 3 hr 11 min: https://hudson.wedontsleep.org/job/db-devel/253/
[05:10] <LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12144
[05:10] <LPCIBot> included.
[08:15] <stub> Anyone know where the code that pulls launchpadlib doctests into the testsuite lives?
[08:18] <lifeless> whats the path of one of these doctests?
[08:52] <stub> eggs/launchpadlib-1.6.5-py2.6.egg/launchpadlib/docs/hosted-files.txt
[08:52] <stub> nm though
[08:52] <stub> Genuine problem, not an issue with the setup environment
[09:07] <stub> Or maybe not... *sigh*
[09:11] <stub> Can we make the current config_fixture a global?
[09:12] <stub> LibrarianFixture is useless without a ConfigFixture. This will pretty much always be BaseLayer.config_fixture. I don't want to use BaseLayer.config_fixture as the default, because that is going circular.
[09:13] <stub> Looking at explicitly passing it in now - not sure how many call sites there are.
[09:23] <stub> Hmm... add_section modifies the generated config, but there is no way of popping those changes...
[09:24] <stub> Maybe would have been better to use the standard push/pop on the config and just get it to write new config files and update LPCONFIG so push/pop also work with subprocesses.
[10:13] <jml> helloall
[10:14] <jml> my space bar isn't responding as well as I would like.
[10:16] <jml> anyone around today?
[10:17] <jelmer_> 'morning jml
[10:17] <jml> jelmer_: hi
[10:18] <jelmer_> it's very quiet here and on the mailing list today
[10:20] <jml> I'll bet.
[10:22] <jml> I'm tackling distributionmirror_prober, trying to make it more like a normal twisted script
[10:22] <jml> so that it leaks resources less
[10:49] <jml> has anyone got a recent subunit log of the full test suite that I could borrow?
[10:54] <jml> ahh, glory. StevenK's hudson builder has the full output
[11:07] <lifeless> stub: yes, that probably would be better
[11:07] <lifeless> stub: given that there was no concept of keeping the necessary fragments, I felt it might be too high a bar to reach in a reasonable timeframe, so I opted for a separate, lesser interface
[11:08] <lifeless> stub: the main config system being hugely complex.
[11:08] <lifeless> stub: I'm interested (but not right now) in why you need to pop stuff out, but certainly you could add that.
[11:10] <stub> So launchpadlib (and old-testing.txt) don't use layers. LaunchpadServerFixture.setUp() doesn't setup everything ready for use. You still need to add the new stanzas to the config, and invoke ConfigUseFixture. I've added the config_fixture as a parameter to LaunchpadServerFixture, and am about to make that also invoke ConfigUseFixture. Which pretty much means ConfigUseFixture is unnecessary, and its functionality moved to ConfigFixture.add_sectio
[11:11] <stub> lifeless: push/pop is more a purity thing. At the moment, a test (or set of tests) that modify the config leave things in a different state than they started.
[11:11] <lifeless> ok
[11:11] <lifeless> that sounds plausible
[11:12] <lifeless> by which I mean - you've clearly looked at it, and nothing there contradicts the overall arch I was working on
[11:12] <lifeless> (AFAICT at this distance)
[11:12] <lifeless> stub: I'd like to nuke layers
[11:12] <StevenK> And I'd like to nuke buildbot, but nothing seems to be happening there
[11:12] <lifeless> stub: but I need to finish the reuse-optimisation support for fixtures + glue them to testresources which has ordering-optimisations.
[11:12] <stub> Sure. Tree has never suited us.
[11:29] <jml> I still don't get why push/pop require names
[11:30] <lifeless> jml: hysterical rasins
[11:31] <jml> lifeless: yeah, but I remember asking about it when the functionality was first added and not getting a clear answer.
[11:31] <lifeless> I think the custom config stuff was engineered with requirements different to our needs.
[11:31] <jml> also, do you know what's awesome? threads.
[11:31] <lifeless> jml: its safer in doctests because they don't [in general] have cleanups
[11:33] <lifeless> and here, gnight all
[11:34] <jml> lifeless: g'night.
[11:36] <stub> jml: Its to make sure you pop what you think you are popping
[11:36] <stub> jml: Otherwise, if code forgets to pop a config later pops succeed and your errors end up in entirely the wrong place.
[11:38] <jml> stub: hmm.
[12:08] <jml> looks like initializing a LaunchpadScript causes bzr functions in the same process to dump to stderr.
[12:36] <stub> jml: That is supposed to be silenced in lib/lp_sitecustomize.py - not sure where the magic that includes that is (buildout.cfg mentions it)
[12:37] <stub> jml: oh... parts/scripts/sitecustomize.py
[12:37] <jml> stub: interesting.
[12:38]  * jml manually copy-pastes those bits into a minimal test case
[12:38] <jml> stub: doesn't work. c.l.scripts.logger blats all over it.
[12:38] <jml> it adds an INFO-level log handler to the root logger
[12:39] <jml> so everything logged at INFO level is printed to stderr
[12:39] <stub> Yes, but the bzr logger should dev null events and not propagate them.
[12:40] <stub> Hmm... which we don't seem to be doing
[12:40] <stub> Unless NullHandler does the magic
[12:44] <jml> back
[12:44] <jml> but stub is gone
[12:44] <jml> oh well.
[13:03] <jml> I think this just leaves lazr.smtptest "No handler" complaints.
[13:06] <jml> (which I have no idea how to address)
[13:07] <jml> anyway, off for lunch
[13:08] <jelmer> bon appetit
[13:12]  * jelmer follows
[14:43] <jml> back again
[14:57] <LPCIBot> Project devel build (339): STILL FAILING in 3 hr 33 min: https://hudson.wedontsleep.org/job/devel/339/
[15:37] <jml> anyone want to review my branch?
[15:37] <jml> no behaviour changes, just moves some code around.
[15:39] <jelmer> jml: Sure
[15:39] <jml> jelmer: https://code.launchpad.net/~jml/launchpad/distribution-mirror-prober/+merge/44656
[15:56] <jml> really starting to hate Python logging.
[15:59] <jelmer> jml: At least it *has* a standard mechanism for logging
[15:59] <jelmer> unlike C where every library has its own custom thing, with different log level names, callbacks, etc
[16:00] <jml> yeah
[16:00] <jml> I guess that's a good thing
[16:02] <jml> btw
[16:02] <jml> always be careful when constructing a LaunchpadCronScript in a test
[16:02] <jml> if you do it wrong, it'll call sys.exit
[16:02] <jelmer> Ouch
[16:03] <jelmer> I bet that can be a bit of a surprise, especially if the output is redirected to something that doesn't go to the terminal.
[16:13] <jml> something is changing the log level too.
[16:17] <jelmer> that reminds me..
[16:17] <jelmer> we added a log handler a couple of months ago that automatically logs OOPSes on Logger.error and Logger.warning.
[16:18] <jml> yes.
[16:18] <jelmer> doing so for warnings isn't really consistent given our policy of treating all OOPSes as high priority bugs IMHO
[16:18] <jelmer> s/given/with/
[16:20] <jml> yeah, quite possibly
[16:49] <jml> https://bugs.launchpad.net/launchpad/+bug/694140
[16:49] <_mup_> Bug #694140: canonical.launchpad.scripts.logger magic breaks bzr logging <Launchpad itself:New> < https://launchpad.net/bugs/694140 >
[16:55] <jelmer> jml: You're creating an old style class in distributionmirror_prober.py
[16:55] <jml> jelmer: hah. so I am.
[16:56] <jelmer> jml: Actually, you're not. the __metaclass__ assignment is just in an unusual place.
[16:57] <jml> easy enough to move
[17:00] <jelmer> jml: It would be nice to have a couple more docstrings - for example, it isn't obvious to me that the "notify_owner" argument to mirror() means notifying the owner when their mirror is disabled
[17:03] <jml> jelmer: fair enough
[17:04] <jelmer> jml: Other than that I'd be happy to +1 it
[17:08] <jml> jelmer: thanks.
[17:08] <jml> jelmer: I've pushed up changes.
[17:17] <jelmer> jml: r=me
[17:18] <lifeless> happy thingy,world
[17:19] <lifeless> jelmer:  on logging -> oopses
[17:19] <lifeless> jelmer: if someone should read it, its worth oopsing
[17:19] <lifeless> jelmer: is the logic that was followed, and I think that thats reasonable
[17:19] <jelmer> lifeless: I agree, but that doesn't mean it's necessarily a high level bug.
[17:19] <jelmer> s/level/priority/
[17:20] <lifeless> jelmer: the question then is 'what logging level means "someone should read it".' ?
[17:20] <lifeless> jelmer: actually, I think it is
[17:21] <lifeless> operational problems drown us out at the moment, and that will only get better once we're on top of it.
[17:22] <jelmer> lifeless: it doesn't help if we just end up with lots of high priority bugs because of this.
[17:22] <lifeless> jelmer: its been active for a month or more now, and we haven't been flooded.
[17:22] <jelmer> lifeless: some calls to Logger.warning have been changed to Logger.info
[17:22] <lifeless> jelmer: and, if we end up fixing lots of production glitchy things because of it, then I argue it would help.
[17:23] <lifeless> jelmer: yes, I saw, and thought those were appropriate choices
[17:23] <lifeless> jelmer: *because* they were things which really didn't need action taken
[17:30] <jelmer> lifeless: hmm
[17:32] <jelmer> lifeless: This makes it harder to distinguish between informational messages and things that look odd but can occur (what I would use warning for)
[17:34] <lifeless> so, ignore logging for a second
[17:34] <lifeless> as a framework
[17:34] <lifeless> we have something running somewhere, with no ui
[17:35] <lifeless> we have a facility you can output high-volume debug stuff that will by default go nowhere
[17:35] <lifeless> a facility that will be logged and you can read on disk
[17:35] <lifeless> and a facility that will generate an OOPS
[17:35] <lifeless> is this insufficient?
[17:37] <lifeless> the first facility is for performance reasons - we can't afford to grab everything every time
[17:37] <jelmer> With the way we currently handle OOPS reports and logs on disk, somewhat.
[17:38] <jelmer> there is no way for something to trigger attention by a human without causing a high level bug report.
[17:38] <lifeless> the second for stuff that we can afford to gather, for helping with bug analysis etc, but which doesn't require an action
[17:38] <lifeless> and the third thing is for triggering human attention
[17:38] <lifeless> jelmer: well, if its not *high* it will sit there and be ignored, based on current stats, for a year or more
[17:39] <lifeless> jelmer: 'needs human attention sometime in 2012' isn't a very useful signal IMO
[17:40] <lifeless> I gotta run. sorry!
[17:40] <jelmer> lifeless: Merry christmas, ttyl!
[17:41]  * jelmer is about to EOD, shops close at 7 today apparantly :-/
[17:43] <jelmer> jml: Merry christmas!
[17:44] <jml> https://bugs.launchpad.net/launchpad/+bug/694152
[17:44] <_mup_> Bug #694152: LaunchpadScript constructor breaks logging for subsequent tests <Launchpad itself:New> < https://launchpad.net/bugs/694152 >
[17:46] <jml> oh
[17:47] <jml> I think https://bugs.launchpad.net/launchpad/+bug/694140 is also why lazr.smtptest keeps generating warnings about no handlers
[17:47] <_mup_> Bug #694140: canonical.launchpad.scripts.logger magic breaks bzr logging <Launchpad itself:New> < https://launchpad.net/bugs/694140 >
[17:49] <jml> lifeless: merry Christmas. You should sleep more :)
[18:39] <jml> Good night all.
[18:39] <jml> Merry Christmas.
[19:28] <LPCIBot> Project db-devel build (254): STILL FAILING in 3 hr 32 min: https://hudson.wedontsleep.org/job/db-devel/254/
[20:28] <LPCIBot> Project devel build (340): STILL FAILING in 3 hr 31 min: https://hudson.wedontsleep.org/job/devel/340/
[23:39] <LPCIBot> Project devel build (341): STILL FAILING in 3 hr 11 min: https://hudson.wedontsleep.org/job/devel/341/
[23:39] <LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=435316] Add tests to make sure Binary: lines
[23:39] <LPCIBot> with newlines are supported in sync_source.
[23:39] <LPCIBot> * Launchpad Patch Queue Manager: [r=bac, jcsackett][ui=sinzui][bug=440406,
[23:39] <LPCIBot> 560701] Help with alternate language usage. List languages on a
[23:39] <LPCIBot> persons translation dashboard.