Calendar:QA Chat:2006-07-20:IRCLog

From MozillaWiki
Jump to: navigation, search
-->| Andreas (Andreas_Tr@F74AD796.4C00166E.33AABD5F.IP) has joined #calendar-qa
|<-- Andreas has left irc.mozilla.org (Quit: ChatZilla 0.9.61 [Mozilla rv:1.7.13/20060417])
-->| Andreas (Andreas_Tr@F74AD796.4C00166E.33AABD5F.IP) has joined #calendar-qa
<ctalbert>    We will be getting under way in about 10 minutes...
-->| zach (zach@moz-46636307.office.mozilla.org) has joined #calendar-qa
<zach>        morning...
<ctalbert>    morning
<zach> 	      ctalbert: the litmus issue is fixed
<ctalbert>    Litmus started working! 
<Andreas>     hello
<ctalbert>    Thanks zach
<ctalbert>    hi andreas
<zach> 	      there's a bit of a bug, which means that whenever you add a  testgroup, I need
              to do something manually in the db (adding subgroups is fine)
<ctalbert>    Ouch.
<ctalbert>    I think I'm through adding testgroups for a while. What about cloning 
              testgroups?
<zach> 	      ctalbert: well you only have one testgroup
<zach> 	      ctalbert: subgroups are unaffected 
<ctalbert>    I was going to flesh out my first testgroup a bit before adding more.
<ctalbert>    That was one of the things I wanted to talk about today.
<zach> 	      if you need to add more testgroups, just let me know once it's added and I 
              can add the needed entry to the db
<ctalbert>    Ok thanks. I will.
<ctalbert>    I guess I should get this started. Marcia should be joining us when she gets in 
              this morning.
<ctalbert>    The tiny agenda is here: 
              http://wiki.mozilla.org/Calendar:Current_QA_Chat
=-= jminta has changed the topic to ``http://wiki.mozilla.org/Calendar:Current_QA_Chat''
-->| ssitter (chatzilla@F41C9EFD.65C15961.3E7AAD02.IP) has joined #calendar-qa
<ctalbert>    my first attempt at a proposed structure for the test groups/subgroups
              and testcases in litmus can be found here:          	 
http://wiki.mozilla.org/Calendar:Test_Case_List#Litmus_Calendar_Product_Test_Case_Structure

<ctalbert>    Ah! more people. Welcome!
<ssitter>     hi, folks
<ctalbert>    The first thing I wanted to get feedback on was a proposed 
              TestGroup/SubGroup/TestCase structure for Litmus. 
<ctalbert>    ssitter: did yo see the wiki link for that?
<ctalbert>    s/yo/you
<ssitter>     yes, some seconds ago
<ctalbert>    I had intended to just point us all directly at litmust to take a look at it,
              but there were some issues there.  The wiki is a little
              more complete anyway.
<ctalbert>    Litmus has three groups for Prodcuts/TestGroups/SubGroups/TestCases 
<ctalbert>    The items in blue are the items that currently exist in Litmus
<ctalbert>    For things that have very generalized testcases (like smoketests
 and 
              localization) I made just one TestGroup for both.
<ctalbert>    Does that seem sufficient?
-->| lilmatt (mwillis@moz-8BBD264E.twcny.res.rr.com) has joined #calendar-qa
<ctalbert>    Hi Matt
<lilmatt>     Hey. Sorry I was late.
-->| marcia (marcia@moz-46636307.office.mozilla.org) has joined #calendar-qa
-->| mvl (michiel@moz-2F035FEA.direct-adsl.nl) has joined #calendar-qa
<ctalbert>    mvl, lilmatt, marcia, I was just talking about a proposal for structuring 
              calendar testcases in litmus here is a link:
<ctalbert> 	http://wiki.mozilla.org/Calendar:Test_Case_List#Litmus_Calendar_Product_Test_Case_Structure
<ctalbert>    It's very simple
<ctalbert>    No one has very many thoughts about it.
-->| marcia_imac (chatzilla@moz-A8CDE328.office.mozilla.org) has joined #calendar-qa
<ctalbert>    One of my principal questions is can we just generalize the local vs. remote 
              calendar tests?
<marcia_imac> ctalbert: dmose is here looking over my shoulder
<ctalbert>    marcia_imac: sweet. Hi dmose
<ctalbert>    local vs. remote calendar: that means we'd have a provider specific suite of 
              testcases for each provider
<ctalbert>    but in our basic functionality testgroup we would direct people to test on any 
              remote calendar
<ctalbert>    does that sound ok?
<ctalbert>    Or am I out of my mind?
* ctalbert    thinks that this should be moved to 16:30UTC so that everyone in the west will 
              be awake :-)
<lilmatt>     Well, as you add providers with diff functionality wouldn't you want to test 
              that?
<lilmatt>     ie: WCAP vs. ICS vs. CalDAV
<marcia_imac> we are awake
<mvl> 	      ctalbert: don't hope to get meaningful input for me, because i never looking 
              into this testcase/litmus thing
<mvl> 	      and besided, i need to go shop for food
<marcia_imac> at least dmose and i are
<ctalbert>    :-)
<zach> 	      I'm awake :)
<ctalbert>    Yes, those would be the provider specific testcases I was mentioned, 
              lilmatt
<lilmatt>     ctalbert: I was late because I had fallen asleep, and I'm EST, so...
<zach> 	      it would seem to me that the remote testcases should test each protocol...
<ctalbert>    Even in the "basic" test group?
<ctalbert>    My main concern was not duplicating too many tests.
	      * ctalbert realizes he forgot the "Provider" SubGroup completely.
<lilmatt>     There's no printing test.
<ctalbert>    Ah, that too.
	      * zach should go over by marcia and dmose
<ctalbert>    So, it would seem I should add in testcases for each provider for the basic 
              functionality. 
<ctalbert>    For example, Create a WebDav calendar would be a testcase
<ctalbert>    I like that since it will provide better tracking per feature.
<lilmatt>     plus whatever spiffy things you can do with each provider.
<ctalbert>    lilmatt: And those spiffy things will go in the Provider specific SubGroup
<lilmatt>     right
<marcia_imac> ctalbert: you can look at some of the Thunderbird test cases to see how we 
              handled the features, esp IMAP and POP
<marcia_imac> ctalbert: ? from dmose, how is the Provider different from calendar?
<ctalbert>    marcia_imac: thanks! I will. 
<ctalbert>    Hmmm....good question
<ctalbert>    I guess it really isn't. It can't exist independently.
<ctalbert>    Thank ya'll. This clears up things quite a bit.
<marcia_imac> ctalbert: besides printing, what other areas will you have for categories 
              in Litmus?
<ssitter>     what about views?
<ctalbert>    Did you see the wiki link? Calendars (Providers), Events, Tasks, Installation,
              Navigation, Searching
<ctalbert>    ssitter: I grouped Views into Navigation
<marcia_imac> ctalbert: No, i will have to check it out
<ctalbert> 	http://wiki.mozilla.org/Calendar:Test_Case_List#Litmus_Calendar_Product_Test_Case_Structure

<ssitter>     drag'n'drop, display, setting view preferences, ...
<jminta>      import/export
<ssitter>     navigation is only one thing you can do in the views
<ctalbert>    ssitter: good point.
<jminta>      ctalbert: is there a plan for how these test groups will line up with either 
              bugzilla or our toronto feature list?
<ctalbert>    jminta: I was thinking about that yesterday and thinking about that.
<ctalbert>    I was thinking that many of the features from Toronto might be going under 
              the "Full Functional Test" bucket rather than the basic, and all I was doing 
              yesterday was trying to flesh out the basic ones
<marcia_imac> ctalbert: i was just mentioning to dmose that our FF and Tbird test cases 
              test basic functionality, but don't emphasize negative tests. I think 
              to have a good test suite you have to incorporate a bit of both
<ctalbert>    I agree
<ssitter>     maybe for mature sunbird: extension/theme installation/handling, 
              update handling, ..
<marcia_imac> i like to think that the BFTs are the most commonly used functions
<ctalbert>    And the FFTs are everything else?
<ssitter>     makes sense
<jminta>      update handling applies to lightning too
<marcia_imac> FFTs for FF and Tbird take a deeper dive, getting into all the things 
              you can do in an area
<lilmatt>     ...and eventually Sb
<ctalbert>    This is very good feedback. I appreciate it.
<ssitter>     jminta: lightning update is handled by thunderbird ;-)
<jminta>      ssitter: we can still screw that up on the server side
<ssitter>     except you want to test server side
<marcia_imac> ctalbert: I think starting with the BFTs is a good thing, and then refining 
              them over time. At least you have a base of tests to start with
<zach> 	      what we've done with ff and tbird is to have the structure for the BFTs 
              and the FFTs be pretty much the same, but to have more testcases 
              in each FFT subgroup
<ctalbert>    that's what I was noticing
<ctalbert>    I think with this feedback I feel a bit more comfortable planning out a 
              complete proposal. Why don't I do that and submit it to m.d.a.calendar
              and we'll let the discussion continue there?
<marcia_imac> We are also trying to write test plans for the upcoming FF 2.0 release. 
              I find that helps me to organize my thoughts and flesh out different 
              ideas
<marcia_imac> ctalbert: sounds like a plan to me
<mvl>         ctalbert: i might be jumping in, and didn't follow the history of this, 
              but would it make sense to couple testcases to the toronto tasks and 
              features?
<ctalbert>    mvl: definitely
<zach> 	      yeah, even just a simple plan for each subgroup laying out the goals 
              of that subgroup is helpful to write to get started
	      * ctalbert nods
<ctalbert>    While we have a bit more time, I'd like to ask marcia if she has any 
              words of wisdom regarding building a qa community around calendar
<mvl> 	      ctalbert: isn't the features list already grouped? and can't you reuse 
              that grouping?
<ctalbert>    mvl: yes, I'll look into that
<marcia_imac> ctalbert: our qa community building is still a work in progress. we have been
              trying different things to see what works best
<ctalbert>    What has seemed to work?
<marcia_imac> ctalbert: things like: blogging and holding regularly scheduled test 
              days. the challenge with that is covering all the time zones
<marcia_imac> the regularly sched test day seem to work well, but we get different 
              contributors all the time. we really need to try to build a base of regular 
              folks. and I would like to involve the comm more in test case writng, etc
<ssitter>     how will the process for writing calendar test cases look like?
<ssitter>     assume i read a bug and think this should be a testcase made of?
<ctalbert>    The test case writing is an interesting thing.
<marcia_imac> ssitter: exactly. sometimes bugs provide good fodder for test cases
<ctalbert>    ssitter: we could get you access to write a test case in litmus. But for 
              people in general, litmus is a bit too young for that at this point.
<marcia_imac> ctalbert: blogging and getting the word out is important to community 
              building.
<ctalbert>    (but they're working on it)
<ctalbert>    I understand.  We could use the calendar blog we have for it. People 
              already know to look there.
<zach> 	      exactly...I'm working on making adminship per-product, so that the calendar 
              product can designate its own admins
<ctalbert>    zach: you're gonna hate this one...do you have a ball park guess for 
              when that will land?
<zach> 	      ctalbert: haha
<zach> 	      ctalbert: maybe late next week?
<ctalbert>    wow. 
<marcia_imac> ctalbert: do you post in the mozillazine forums at all announcing things?
<ctalbert>    Not at the moment, no
<ctalbert>    We should.
<zach> 	      ctalbert: depends how long it takes me to untangle myself from perl module 
              dependency hell ;)
<marcia_imac> ctalbert: good set of folks there to harvest. We post to the QA blog, 
              MozillaZine forums and the mozilla newsgroups
<ctalbert>    Ok. Thanks that's good to know. We can do the same and include our calendar 
              specific stuff that we've got going.
<marcia_imac> ctalbert: it seems to me that the forums are actually a better place 
              to try to get new recruits. especially for things related to Tbird
<ctalbert>    marcia_imac: How "complete" were your testcases before you did the first 
              test day? 
<marcia_imac> ctalbert: that sounds good.
<marcia_imac> ctalbert: how often do you guys blog? daily or weekly? i think frequent 
              blogging is helpful too. I know we do not do enough of it
<jminta>      about weekly
<ssitter>     (http://weblogs.mozillazine.org/calendar/)
<zach> 	      ctalbert: even a small suite of tests is helpful
<ctalbert>    I was thinking of having one when we got the first round of BFTs fleshed out.
<ctalbert>    (for Sunbird)
<marcia_imac> ctalbert: they were already fairly complete, because we really wrote 
              most of them in the FF 1.0 days. I think waiting until you at least have 
              test cases for each section is a good idea
<zach> 	      jminta has a good idea: hold a test writing day
<marcia_imac> what about also having a test day focused around one feature? we are doing 
              that for FF 2.0
<zach> 	      people could write tests on the wiki or someplace and send them to ctalbert 
              for editing and insertion in litmus
<ctalbert>    Kind of like around jminta's drag and drop tuff that just landed.
<ctalbert>    zach: That's how I was thinking of doing it.
<marcia_imac> ctalbert: for example, with FF 2.0 appraching, we are zeroing on on RSS 
              and Session Restore, which we want to flesh out the problems early before 
              we ship
<marcia_imac> ctalbert: drag and drop stuff would make a great test day
<ctalbert>    zach: In fact, if you scroll down that wiki page, that's exactly what it is.
<zach> 	      ctalbert: awesome
<ssitter>     i did a test day for that dnd landing, jminta will remember i'm sure ;-)
<marcia_imac> ctalbert: how you handle feedback for the test days is important too, we open 
              up the blog for comments too in addition to the test days
<ctalbert>    I have one more thing before we wrap up.
<zach> 	      ctalbert: a test writing day would be a good way to build up enough testcases 
              for test days
<marcia_imac> also rewarding folks on an ongoing basis is a good thing - maybe I can 
              help out with that
<ctalbert>    marica_imac: good_idea. Thanks
<ctalbert>    my other thing: If I use this timeslot again for a meeting or a test 
              day or whatever, would it be better for people to move it back 30mins?
<marcia_imac> jminta said there is a sunbird logo, maybe we could make a shirt
<ctalbert>    16:30 UTC for example
<ctalbert>    Oh...that would be cool
<marcia_imac> ctalbert: that works better for me, dmose is mentioning that the sun 
              guys might need to weigh in there
<ssitter>     yes, 16:00 is hard to make for me cause of my day job
<ctalbert>    We have the sun guys who are at their day job at 16:00 and guys like 
              ssitter and mvl who could use a slightly later time
<ctalbert>    I'll post that to the newsgroup as well
<ssitter>     16:30 utc would be better
<Andreas>     I think it would be ok for Ulf and me (The Sun QA) to start the chat 
              30 minutes later.
<ctalbert>    Awesome. Then I think we'll do that next week. 
<ctalbert>    Ok, I think we're out of time. I'll send out a complete test-plan proposal 
              to the m.d.a.calendar newsgroup
<ctalbert>    Then I think we'll try to have a test-writing day to help get some of 
              these things written
<marcia_imac> ctalbert: sounds good to me. let me know if you have any other questions 
              via email or ping me on irc
<ctalbert>    Thanks, I will.
<ctalbert>    Thank you all for your help.
<marcia_imac> ctalbert: no problem
<ssitter>     ctalbert: maybe you add definition what is/should be difference between 
              smoketest, basic test, full test
<ctalbert>    I'll be posting this log on the calendar qa chat wiki
<ctalbert>    ssitter: good idea.
<ctalbert>    I will
<ctalbert>    Ok. Meeting adjourned. Have a good day.
<ctalbert>    zach: Thanks for your help with litmus!
<zach> 	      ctalbert: certainly! coop says the bug will be fixed soon, so you'll 
              be able to add your own testgroups
<lilmatt>     marcia_imac: Could I get some assistance from you in getting template 
              talkback master.ini files for us to make for Sunbird?
<lilmatt>     I don't believe they're in public cvs
<ctalbert>    brb
<marcia_imac> lilmatt: i think jay patel might be able to help better in that area
<ctalbert>    all, I am going to have to step out for a while. I'll be back in about 
              an hour.
<lilmatt>     Okay. His email to me had the following regarding configuring templates
              for Sunbird : "I can help you with this if you know where the Talkback 
              files are."
=-= YOU are now known as ctalbert|away
<--| zach has left #calendar-qa
=-= lilmatt is now known as lilmatt-shower
<--| ssitter has left #calendar-qa
<--| jminta has left #calendar-qa
<-- Andreas has left irc.mozilla.org (Quit: ChatZilla 0.9.61 [Mozilla rv:1.7.13/20060417])
-= YOU are now known as ctalbert
--| mvl has left #calendar-qa (Leaving)