Calendar:QA Chat:2006-10-19:Log

From MozillaWiki
Jump to: navigation, search
<ctalbert>	It's about 16:30UTC.
<ctalbert>	Shall we get started?
<Sebo>	oh yeah!
<ctalbert>	sebo has coffee!
<ctalbert>	hah.
<Sebo>	my constant drug
<ctalbert>	The test day results are up: http://wiki.mozilla.org/index.php?title=Calendar:QA_TestDay:2006-10-17&action=submit
<ctalbert>	oops
<ctalbert>	saving the page
<ctalbert>	...
<ctalbert>	Now they're up: 
http://wiki.mozilla.org/Calendar:QA_TestDay:2006-10-17
<ctalbert>	I've made some notes about the day there in my writeup, does
 anyone have other points they would like to make it?
<ctalbert>	s/it/about it?
<Sebo>	about bug 356967
<firebot>	Sebo: Bug https://bugzilla.mozilla.org/show_bug.cgi?id=356967 
nor, --, ---, nobody@mozilla.org, NEW, Importing .ics file into Thunderbird 
using lightning-0.3 does not register
<ctalbert>	Yes?
<Sebo>	I think it has to do with times containing the Z at the end
<Sebo>	I had a similar problem with Google Calendar
<ctalbert>	That should designate the time is in UTC time.
<ctalbert>	That's interesting.
<Sebo>	I noted it in the table but didnt have time to search bugzilla yet
<ctalbert>	I did a bit of searching, didn't see anything that seemed quite 
like it. The ICS file he uses is very tiny, which I thought would make a good, 
easily reproducible testcase. That's why I confirmed it.
<ctalbert>	And of course, I confirmed it on my sunbird too
<Sebo>	ok, I will add my Google calendar, maybe its the same problem
<ctalbert>	Good idea.
<ctalbert>	Jamesb, I'll get you the email addresses for those people after 
the meeting, so you can send out the rewards.
<ctalbert>	Anything else on the Test Day?
<ctalbert>	Ok. I'll be happy to talk about it anytime, if you think of
 something later. Moving on to 0.3. First off, I want to update ya'll on the 
download numbers.
<ctalbert>	As of yesterday, there have been 54,342 downloads of Sunbird 
since 0.3 release!
<ctalbert>	There have been 5,045 downloads of Lightning just from 
addons.mozilla.org since 0.3 release!
<ctalbert>	This is really good news. It means that people are really quite 
interested in what we're working on and what we're putting 
together.
<ctalbert>	We came together as a calendar-qa team late in the 0.3 Release
 process.
<ctalbert>	Even so, we did a lot of good work.
<ctalbert>	I'd be interested to hear people's thoughts on what went well 
during the 0.3 release, and what didn't, what we can do 
better for the 0.5 release.
<ctalbert>	The big criticism from the conference calls about QA was that 
the QA came in too late for the release and the testing
 was not deep enough during the release cycle.
<lilmatt>	the flip side of that is that we HAD a QA team
<jminta>	which rocks
<ctalbert>	Exactly. And that QA team didn't exist until August.
<ctalbert>	Which was right when we were supposed to release.
<ctalbert>	So, I don't really count that criticism as totally valid.
<ctalbert>	But, I think it's important to know what the 
perception outside this little QA community is.
<Sebo>	So what we could do about testing in release cycle
<ctalbert>	Yes, that's the question.
<ctalbert>	Having test days every two weeks is definitely a good start.
<Sebo>	or to have a test suite. and each release candidate needs to pass
<ctalbert>	^^^ that's a very good idea.
<ctalbert>	I think we should put that together
<ctalbert>	During the upcoming test days, I'm going to make it my 
personal priority to engage the people that come in better,
 and try to help get more testing accomplished in
 those days as well as possibly recruit those people to
 help out at other times too.
<ctalbert>	So, a comprehensive test suite for release candidates
 to pass, and better engagement in test days. What else 
can we do?
<ctalbert>	I think it might be useful to blend our Calendar 
QA Todo list with the test days.
<celina63>	I would like to see some sort of automated testing
<ctalbert>	For example, the Calendar QA ToDo list would 
be valid from TestDay to Testday, so from Oct 17 to Oct 31. 
Then after the test day, we can put together another ToDo List
<celina63>	Clint, that makes sense... no duplication of effort
<ctalbert>	Cool.
<ctalbert>	Let's talk about automated testing. jminta and I 
are going to be working on two test tools in the coming weeks. 
What do you forsee as the goals of the automated testing?
<celina63>	same thing I use it for here... smoke and regression
<ctalbert>	That was my idea as well.
<Sebo>	couldnt import export be done automatically
<celina63>	that will allow us to focus on the not-so-common
<ctalbert>	Yes, it could.
<celina63>	but still give us code coverage
<celina63>	does mozilla have open-source automated test tools already?
<ctalbert>	Yes and no. There are some that exist. Some that
 are in development.
<ctalbert>	Most of them are tailored for our XPCOM/XPConnect/
JavaScript/XUL environment
<ctalbert>	There is not one set of "blessed" automated test 
tools at the moment.
<jminta>	i have kinda a developer preview at http://pantheon.yale.edu/~jpm76/calQA.xpi
<ctalbert>	Cool.
<jminta>	it's not perfect yet, but it ought to be somewhat usable
<jminta>	and it's only for Sunbird at the moment
<celina63>	what is an .xpi file?
<jminta>	an extension
<jminta>	Tools->Addons
<celina63>	lol
<celina63>	oh ok
<celina63>	dur
<ctalbert>	The technology used to be called XPInstall
<ctalbert>	hence the .xpi
<celina63>	until that pesky Windows XP came out? :D
<ctalbert>	jminta's QA tool will be interactive and make use 
of the GUI for verification. The one I'll work on will be a fully 
automated suite that feeds calendar data into our low level 
provider infrastructure.
<celina63>	awesome
<ctalbert>	I'll get a design for it up on the wiki here either today or tomorrow.
<ctalbert>	I've got another point I want to ask about. 
These meetings. The "QA Chat". Should anything be changed about it?
<jminta>	ctalbert: ping me about that, i have a couple 
provider tests wired into that extension
<ctalbert>	I will
<ctalbert>	Do you want to try a conference call? Do I type too much?
<ctalbert>	Should we have them every two weeks?
<ctalbert>	Are these Chats helpful?
<Sebo>	jminta: testing rocks with this tool. (4 failures btw)
<ctalbert>	ha!
<Sebo>	maybe we shoud concentrate more to talk about bugs during the chat?
<ctalbert>	We can do that.
<ctalbert>	Like unconfirmed bugs?
<Sebo>	going over the qawanted list
<jminta>	Sebo: i think 4 is the expected number right now
<Sebo>	or the unconfirmed
<ctalbert>	Sebo: I think that's a great idea.
<Sebo>	we might run out of time easily, though
<ctalbert>	That's ok.
<ctalbert>	If we run out of time, we'll just update the QA 
ToDo list with what else needs to be addressed.
<celina63>	I think doing the chat this way is fine...
 a conference call I can't support (or log, for that matter)
<ctalbert>	It would be easier to discuss bugs in this format too.
<celina63>	my *real* job would have an issue with me being 
on the phone for non-business purposes during business hours :D
<ctalbert>	Thanks, this is good feedback. Feedback is welcome 
anytime, by the way.
<ctalbert>	We're sort of covering items out of order on the 
agenda. One last thing, and then we'll do a quick discussion of
 qawanted bugs.
<ctalbert>	We have the following goals for 0.5:
<ctalbert>	Test days every two weeks
<ctalbert>	test suite for the RC before it can be released
<ctalbert>	test automation tools
<ctalbert>	using this time to talk more about qawanted list
<ctalbert>	Anything else?
<Sebo>	one other thing:
<Sebo>	Sometimes I dont know what the app is supposed to be doing
<Sebo>	So it would be nice to have a feature spec
<Sebo>	example: drag and drop recurring event
<ctalbert>	I was just about to say that
<Sebo>	or inline edit recurring event
-->|	nightrat (nightrat@moz-61E91F7D.pools.arcor-ip.net) has joined #calendar-qa
<ctalbert>	Okay, Let's see if we can put that together. 
Dmose would be the best person to write something like that.
<celina63>	ooo a new features list
<ctalbert>	I'll start a feature spec on the wiki, and we'll 
collectively bug people to update it for us.
<ctalbert>	I'll put everything I know into it.
<ctalbert>	lilmatt, jminta, it would be good if ya'll could 
help with this too.
<ctalbert>	I also might be able to persuade ssitter to help 
update it. He knows a ton about how everything is supposed 
to work
<ctalbert>	Alright, then in the spirit of change, let's 
discuss the QAwanted list in the last 10 mins or so that we have.
<ctalbert>	http://tinyurl.com/srdgc
<Sebo>	ok, there is this October 31 bug. We talked about this already
<ctalbert>	Yes, I thought we had confirmed that thing
<Sebo>	not this one especially
<ctalbert>	oh that was the other one
<Sebo>	we still need the system timezone of the reporter
<Sebo>	I could confirm the other two that are related (see last comment)
<ctalbert>	I see that. And until we get the system timezone
 of the reporter, I don't think there's any more QA needed 
here. Do you?
<Sebo>	agrreed, apart from the fact that I have no idea what causes this
<Sebo>	note that the year is important
<Sebo>	it doesnt happen every october 31
<ctalbert>	It has something to do with the date of the daylight
 savings time change in that year, I think
<Sebo>	but this is all we can do (right?)
<ctalbert>	I think so. Our job is to figure out all we can about
 what other bugs its related to, get a reproducible test case 
(if possible), and provide any other information we can to the 
developers. You've certainly done all of that.
<ctalbert>	The developers get the fun of figuring out why it happens.
* ctalbert	removes qawanted
<Sebo>	I got the mail :-)
<ctalbert>	There are a bunch of unconfirmed items on here.
<Sebo>	I added something to bug 347128, but it is still unclear
<firebot>	Sebo: Bug https://bugzilla.mozilla.org/show_bug.cgi?id=347128
 nor, --, ---, nobody@mozilla.org, UNCO, Changes to a remote calendar
 using ftp (without providing the username/password in the URL) are 
not
<ctalbert>	jminta: Do you feel like sebo's comment addresses
 your questions? Do you need anything else done on this bug? ^^^^
<ctalbert>	brb
<ctalbert>	Hmm...no jminta
<Sebo>	I didnt look at any other qawanted bugs last week...
<ctalbert>	I'm not sure what question Joey hoped to get answered, 
so I'm not sure how to make the call on this one, Sebo
<Sebo>	thats fine.
<firebot>	ssitter@googlemail.com set the Resolution field on 
bug 357281 to DUPLICATE of bug 353070.
<firebot>	Bug https://bugzilla.mozilla.org/show_bug.cgi?id=357281 
nor, --, ---, nobody@mozilla.org, RESO DUPLICATE, Moving a task 
without 'due to' date to another date with 'drag&drop' method opens an edit window
<ctalbert>	Let's leave that one until we hear from him.
<firebot>	Bug https://bugzilla.mozilla.org/show_bug.cgi?id=353070
 nor, --, ---, nobody@mozilla.org, NEW, drag and drop for tasks do 
not work if due date is not set
<ctalbert>	It looks like we're out of time too.
<ctalbert>	Sebo, do you want to take the lead on creating a test
 suite for a RC to pass?
<ctalbert>	If you could start a page with ideas on the wiki, we 
can all jump in and help out
<Sebo>	OK, I can try. Wouldnt this coincide with litmus testcases?
<ctalbert>	I'll start one for the "features list" and the automation 
design
<ctalbert>	It would. I mean, the entire test suite might be a set 
of litmus test cases.
<Sebo>	ok
<ctalbert>	That would be fine.
<ctalbert>	In fact, that would be ideal.
<ctalbert>	But, there are probably tests that we should run that 
aren't in litmus yet.
<ctalbert>	Ok. And unless anyone has anything else, the "official"
 meeting is over.
<ctalbert>	Thanks for the help and the feedback!
<Sebo>	ok, see you...
<ctalbert>	later!
|<--	Sebo has left irc.mozilla.org 
(Quit: Chatzilla 0.9.75 [Firefox 2.0/2006101803])
* celina63	will attempt to check out the automated tools later today
<ctalbert>	that would be great. I have some URLs to start 
you off on your search.
<ctalbert>	http://www.mozilla.org/scriptable/XPCShell.html
<celina63>	wooo
<celina63>	:)
<ctalbert>	http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page#Automation
<ctalbert>	Hmmm thought I had more stuff than that. I guess not.
 Another source for Mozilla automation information is the 
mozilla.dev.quality (the Mozilla QA newsgroup).