As promised I'm starting catching up on blogging after being sick and traveling. Here's my report of what I saw and found interesting at this year's FOSDEM which was held the last weekend in Brussels.
On Friday evening I've tried to attend the FOSDEM beer event at Delirium Cafe but had a bad luck. At 21:30 the place was already packed with people. I managed to get access to only one of the rooms but it looked like the wrong one :(. I think the space is definitely small for all who are willing to attend.
During Saturday morning I did a quick sight-seeing most notably Mig's wine shop and the area around it since I've never been to this part of the city before. Then I took off to FOSDEM arriving at noon (IOW not too late).
I've spent most of my day at building K where project stands were and I stayed quite a long time around the Python and Perl stands meeting new people and talking to them about their upgrade practices and how they manage package dependencies (aka promoting Difio).
Fedora stand was busy with 3D printing this year. I've seen 3D printing before but here I was amazed of the fine-grained quality of the pieces produced. This is definitely something to have in mind if you are building physical products.
Red Hat's presence was very strong this year. In addition to the numerous talks they gave there were also oVirt and OpenShift Origin stands which were packed with people. I couldn't even get close to say hi or take a picture.
Near the end of the day I went to listen to some of the lightning talks. The ones that I liked the most were MATE Desktop and DoudouLinux.
The thing about MATE which I liked is that they have a MATE University initiative which is targeting developers who want to learn how to develop MATE extensions. This is pretty cool with respect to community and developers on-boarding.
DoudouLinux is a Debian based distribution targeted at small children (2 or 3 years old) based on simple desktop and educational activities. I've met project leader and founder Jean-Michel Philippe who gave the talk. We chatted for a while when Alejandro Simon from Kano came around and showed us a prototype of their computer for children. I will definitely give DoudouLinux a try and maybe pre-order Kano as well.
In the evening there was a Python beer event at Delirium and after that dinner at Chez Leon where I had snails and rabbit with cherries in cherry beer sauce. I've had a few beers with Marc-Andre from eGenix and Charlie from Clark Consulting and the talk was mostly about non-technical stuff which was nice.
After that we went back to Delirium and re-united with Alexander Kurtakov and other folks from Red Hat for more cherry beer!
Report of the second day of FOSDEM'14 on Sunday is here.
There are comments.
Last week I've expressed my thoughts about the state of upstream test suites in Fedora along with some other ideas. Following the response on this thread I'm starting to analyze all SRPM packages in Fedora 20 in order to establish a baseline. Here are my initial findings.
I've found two source distributions for Fedora 20:
Fedora-20-source-DVD.iso
file which to my knowledge contains the sources
of all packages that comprise the installation media;Everything/source/SRPMS/
directory which appears to contain the sources
of everything else available in the Fedora 20 repositories.There are 2574 SRPM packages in Fedora-20 source DVD and 14364 SRPMs in the Everything/ directory. 9,2G vs. 41G.
Fedora Packaging Guidelines state
If the source code of the package provides a test suite, it should be executed in the %check section, whenever it is practical to do so.
In my research I found 738 SRPMs on the DVD which have a %check
section and 4838 such packages under Everything/
. This is 28,6% and 33,6%
respectively.
A quick grep for either test/
or tests/
directories in the package sources revealed
870 SRPM packages in the source DVD which are very likely to have a test suite.
This is 33,8%. I wasn't able to inspect the Everything/
directory with this script
because it takes too long to execute and my system crashed out of memory.
I will update this post later with that info.
UPDATE 2014-01-02:
In the Everything/
directory only 4481 (31,2%) SRPM packages appear to have
test suites.
The scripts and raw output are available at https://github.com/atodorov/fedora-scripts.
So it looks like on average 30% of the packages execute their test suites at build time in the %check section and less than 35% have test suites at all! There's definitely room for improvement and I plan to focus on this during 2014!
There are comments.
Today I had troubles with my Omnikey CardMan 6121 smart card reader.
For some reason it will not detect the card inside and was unusable.
/var/log/messages
was filled with Card Not Powered messages:
Dec 18 11:17:55 localhost pcscd: eventhandler.c:292:EHStatusHandlerThread() Error powering up card: -2146435050 0x80100016
Dec 18 11:18:01 localhost pcscd: winscard.c:368:SCardConnect() Card Not Powered
Dec 18 11:18:02 localhost pcscd: winscard.c:368:SCardConnect() Card Not Powered
I've found the solution in RHBZ #531998.
I've found the problem, and it's purely mechanical. Omnikey has simply screwed up when they designed this reader. When the reader is inserted into the ExpressCard slot, it gets slightly compressed. This is enough to trigger the mechanical switch that detects insertions. If I jam something in there and force it apart, then pcscd starts reporting that the slot is empty.
Pierre Ossman, https://bugzilla.redhat.com/show_bug.cgi?id=531998#c12
So I tried moving the smart card a millimeter back and forth inside the reader and that fixed it for me.
This smart card is standard SIM size and I wonder if it is possible to use dual SIM smart phones and tablets as a reader? I will be happy to work on the software side if there is an open source project already (e.g. OpenSC + drivers for Android). If not, why not?
If you happen to have information on the subject please share it in the comments. Thanks!
There are comments.
Accidentally I have discovered a bug for Python's URLGrabber module which has to do with change in behavior in libcurl.
>>> from urlgrabber.grabber import URLGrabber
>>> g = URLGrabber(reget=None)
>>> g.urlgrab('https://s3.amazonaws.com/production.s3.rubygems.org/gems/columnize-0.3.6.gem', '/tmp/columnize.gem')
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/celeryd/.virtualenvs/difio/lib/python2.6/site-packages/urlgrabber/grabber.py", line 976, in urlgrab
return self._retry(opts, retryfunc, url, filename)
File "/home/celeryd/.virtualenvs/difio/lib/python2.6/site-packages/urlgrabber/grabber.py", line 880, in _retry
r = apply(func, (opts,) + args, {})
File "/home/celeryd/.virtualenvs/difio/lib/python2.6/site-packages/urlgrabber/grabber.py", line 962, in retryfunc
fo = PyCurlFileObject(url, filename, opts)
File "/home/celeryd/.virtualenvs/difio/lib/python2.6/site-packages/urlgrabber/grabber.py", line 1056, in __init__
self._do_open()
File "/home/celeryd/.virtualenvs/difio/lib/python2.6/site-packages/urlgrabber/grabber.py", line 1307, in _do_open
self._set_opts()
File "/home/celeryd/.virtualenvs/difio/lib/python2.6/site-packages/urlgrabber/grabber.py", line 1161, in _set_opts
self.curl_obj.setopt(pycurl.SSL_VERIFYHOST, opts.ssl_verify_host)
error: (43, '')
>>>
The code above works fine with curl-7.27 or older while it breaks with curl-7.29 and newer. As explained by Zdenek Pavlas the reason is an internal change in libcurl which doesn't accept a value of 1 anymore!
The bug is reproducible with a newer libcurl version and a vanilla urlgrabber==3.9.1 from PyPI (e.g. inside a virtualenv). The latest python-urlgrabber RPM packages in both Fedora and Amazon Linux already have the fix.
I have tested the patch proposed by Zdenek and it works for me. I still have no idea why there aren't any updates released on PyPI though!
There are comments.
In the last few weeks I've been working together with Tim Flink and Kamil Paral from the Fedora QA team on bringing some installation testing expertise to Fedora and establishing an open source test lab to perform automated testing in. The infrastructure is already in relatively usable condition so I've decided to share this information with the community.
Beaker is the software suite that powers the test lab infrastructure. It is quite complex, with many components and sometimes not very straight-forward to set up. Tim has been working on that with me giving it a try and reporting issues as they have been discovered and fixed.
In the process of working on this I've managed to create couple of patches against Beaker and friends. They are still pending release in a future version because of more urgent bug fixes which need to released first.
SNAKE is a client/server Python framework used to support Anaconda installations. It supports plain text ks.cfg files, IIRC those were static templates, no variable substitution.
The other possibility is Python templates based on Pykickstart:
from pykickstart.constants import KS_SCRIPT_POST
from pykickstart.parser import Script
from installdefaults import InstallKs
def ks(**context):
'''Anaconda autopart'''
ks=InstallKs()
ks.packages.add(['@base'])
ks.clearpart(initAll=True)
ks.autopart(autopart=True)
script = '''
cp /tmp/ks.cfg /mnt/sysimage/root/ks.cfg || \
cp /run/install/ks.cfg /mnt/sysimage/root/ks.cfg
'''
post = Script(script, type=KS_SCRIPT_POST, inChroot=False)
ks.scripts.append(post)
return ks
At the moment SNAKE is essentially abandoned but feature complete. I'm thinking about adopting the project just in case we need to make some fixes. Will let you know more about this when it happens.
I have been working on opening up several test cases for what we (QE) call a tier #1 installation test suite. They can be found in git. The tests are base on beakerlib and the legacy RHTS framework which is now part of Beaker.
This effort has been coordinated with Kamil as part of a pilot project he's responsible for. I've been executing the same test suite against earlier Fedora 20 snapshots but using an internal environment. Now everything is going out in the open.
Well you can't do that - YET! There are command line client tools for Fedora but Beaker and SNAKE are not well suited for use outside a restricted network like LAN or VPN. There are issues with authentication most notably for SNAKE.
At the moment I have to ssh through two different systems to get proper access. However this is been worked on. I've read about a rewrite which will allow Beaker to utilize a custom authentication framework like FAS for example. Hopefully that will be implemented soon enough.
I will also like to see the test systems have direct access to the Internet for various reasons but this is not without its risks either. This is still to be decided.
If you are interested anyway see the kick-tests.sh
file in the test suite for
examples and command line options.
The first successfully completed test jobs are jobs 50 to 58. There's a failure in one of the test cases, namely SELinux related RHBZ #1027148.
From what I can tell the lab is now working as expected and we can start doing some testing against Fedora development snapshots.
Ping me or join #fedora-qa on irc.freenode.net if you'd like to join Fedora QA!
There are comments.
Today I was looking for a command sequence to cut a string in two by predefined
delimiter (e.g. like cut
does). I wanted to get the last field only and all
fields but the last as separate variables.
The proposed solutions I've found suggested using awk
but I don't like it.
Here's a simple solution using cut
and rev
which can extract arbitrary
field counts from the end of the string.
$ echo 'buildvm-08.phx2.fedoraproject.org' | rev | cut -f1 -d. | rev
org
$ echo 'buildvm-08.phx2.fedoraproject.org' | rev | cut -f-2 -d. | rev
fedoraproject.org
$ echo 'buildvm-08.phx2.fedoraproject.org' | rev | cut -f-3 -d. | rev
phx2.fedoraproject.org
$ echo 'buildvm-08.phx2.fedoraproject.org' | rev | cut -f2- -d. | rev
buildvm-08.phx2.fedoraproject
$ echo 'buildvm-08.phx2.fedoraproject.org' | rev | cut -f3- -d. | rev
buildvm-08.phx2
$ echo 'buildvm-08.phx2.fedoraproject.org' | rev | cut -f4- -d. | rev
buildvm-08
The magic here is done by rev
which reverses the order of characters in every
line. It comes with the util-linux-ng package.
Note to Self: util-linux-ng appears to contain more useful commands which I wasn't aware of. Need to RTFM a little bit.
There are comments.
Consider the following scenario:
Just yesterday I hit an issue with the above setup and figured Fedora QA is in a kind of strange situation - we always need the latest but need it conservative enough to run on RHEL 6. See the original thread at kickstart-list.
In this particular case the solution will be to remove the offending code and implement the same functionality in backward-compatible manner. Also add more tests. I will be working on this tomorrow (there's an older patch already).
The BIG question remains though - how do you manage software evolution and still keep it compatible with older execution stacks? Please share your experience in the comments section.
PP: Spoiler - this is part of an ongoing effort to bring open source installation testing expertise (my domain) into Fedora world, plus establish a community supported test infrastructure. More info TBA soon.
There are comments.
image by opensource.com
Recently OpenSource.com published my first article User guide for open source project bug submissions which was inspired by real events.
The comments at the bottom are very interesting. It looks like Fedora needs to make some changes in the bug reporting process and tools.
I've started a discussion on Fedora QA mailing list about this. Follow the thread if you are interested in the topic.
There are comments.
I was testing Fedora 20 inside a KVM guest this week when the disk space run out. The system was configured to use Btrfs filesystem and this is how to extend it.
First you have to extend the underlying guest storage. On the host I'm using LVM so this is a no brainer:
# pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/luks-f3f6cea1-baba-4aaf-bca8-33a0ec540369 vg_redbull_mini lvm2 a-- 289,11g 134,11g
# lvs
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
vm_fedora vg_redbull_mini -wi-ao--- 15,00g
# lvextend -L +5G /dev/mapper/vg_redbull_mini-vm_fedora
Extending logical volume vm_fedora to 20,00 GiB
Logical volume vm_fedora successfully resized
# lvs
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert
vm_fedora vg_redbull_mini -wi-ao--- 20,00g
# pvs
PV VG Fmt Attr PSize PFree
/dev/mapper/luks-f3f6cea1-baba-4aaf-bca8-33a0ec540369 vg_redbull_mini lvm2 a-- 289,11g 129,11g
On the VM we have a default Btrfs layout:
# blkid
/dev/vda1: UUID="410ee563-e701-42ff-9d5f-5805dd103e35" TYPE="ext4" PARTUUID="0000330f-01"
/dev/vda2: UUID="f4addad4-a0fc-482e-ad5a-240864b76f09" TYPE="swap" PARTUUID="0000330f-02"
/dev/vda3: LABEL="fedora" UUID="f0b589ce-061c-4ac3-826e-7f3f8c8a6d30" UUID_SUB="11aa8414-3ce1-4fe7-a506-9a4f91ba5c30" TYPE="btrfs" PARTUUID="0000330f-03"
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda3 13G 11G 1.4G 89% /
devtmpfs 996M 0 996M 0% /dev
tmpfs 1002M 80K 1002M 1% /dev/shm
tmpfs 1002M 668K 1002M 1% /run
tmpfs 1002M 0 1002M 0% /sys/fs/cgroup
tmpfs 1002M 16K 1002M 1% /tmp
/dev/vda3 13G 11G 1.4G 89% /home
/dev/vda1 477M 72M 376M 17% /boot
Now power-off (not reboot) and power-on the VM guest so that it sees the new size of the underlying storage. See the fdisk header (line 9 below), vda is now 20GiB!
Before extending the filesystem you have to extend the underlying disk partition! This is the trickiest part. Using fdisk or parted you have to delete the partition and add it again. Make sure to use the SAME starting sector for the new partition (line 33)!
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | # fdisk /dev/vda
Welcome to fdisk (util-linux 2.24-rc1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): p
Disk /dev/vda: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0000330f
Device Boot Start End Blocks Id System
/dev/vda1 * 2048 1026047 512000 83 Linux
/dev/vda2 1026048 5253119 2113536 82 Linux swap / Solaris
/dev/vda3 5253120 31457279 13102080 83 Linux
Command (m for help): d
Partition number (1-3, default 3): 3
Partition 3 is deleted
Command (m for help): n
Partition type:
p primary (2 primary, 0 extended, 2 free)
e extended
Select (default p): p
Partition number (3,4, default 3): 3
First sector (5253120-41943039, default 5253120):
Last sector, +sectors or +size{K,M,G,T,P} (5253120-41943039, default 41943039):
Created a new partition 3 of type 'Linux' and of size 17,5 GiB.
Command (m for help): p
Disk /dev/vda: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0000330f
Device Boot Start End Blocks Id System
/dev/vda1 * 2048 1026047 512000 83 Linux
/dev/vda2 1026048 5253119 2113536 82 Linux swap / Solaris
/dev/vda3 5253120 41943039 18344960 83 Linux
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Device or resource busy
The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).
# partprobe
Error: Partition(s) 3 on /dev/vda have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should reboot now before making further changes.
# reboot
|
See lines 36 and 49 above. The new partition has a greater size. After reboot just resize the filesystem and verify the new space has been added
# btrfs filesystem resize max /
Resize '/' of 'max'
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda3 18G 11G 6.4G 63% /
devtmpfs 996M 0 996M 0% /dev
tmpfs 1002M 80K 1002M 1% /dev/shm
tmpfs 1002M 660K 1002M 1% /run
tmpfs 1002M 0 1002M 0% /sys/fs/cgroup
tmpfs 1002M 16K 1002M 1% /tmp
/dev/vda3 18G 11G 6.4G 63% /home
/dev/vda1 477M 72M 376M 17% /boot
This is it, more disk space available for the virtual machine. Let me know how it works for you.
There are comments.
Every once in a while ABRT will tell you that reporting is disabled because backtrace is unusable. What it means is that it can't read some of the debugging symbols and the most likely reason for that is debuginfo packages are missing.
To install them first locate the directory containing the files for that particular
crash. Use the executable
file to find out if you are looking into the correct
directory. Then use this one liner to install the missing debuginfo packages.
# pwd
/var/tmp/abrt/ccpp-2013-10-10-15:55:18-15533
# cat backtrace | grep lib | tr -s ' ' | cut -f4 -d' ' | sort | uniq | grep "/" | xargs rpm -qf --qf "%{name}\n" | xargs debuginfo-install -y
There are comments.
Here is my summary of the second Fedora Test Day hosted at init Lab yesterday.
Local attendance was a total disaster, in fact I was testing once again by my own. This time there were more people in the lab, all busy with their daily routines and tasks. There were no people who came for the testing :(. I will have to try different venues in the future and see if the situation improves.
On the testing front I managed to score 5 bugs against GNOME and Fedora. You can see the other test results and bugs on the wiki.
There are two things I didn't like in particular
At one time I was engaged in a discussion about which Bulgarian keyboard layout should be the default in GNOME simply because of GNOME #709799. The default keyboard layout will be Bulgarian (traditional phonetic) aka bg+phonetic.
(16,13,26) rtcm: atodorov: are you bulgarian and/or live in bulgaria?
(16,14,20) rtcm: atodorov: if yes, I wanted to know which keyboard layout most people expect there to be the default
(16,16,34) atodorov: rtcm: I'm a Bulgarian, however I can't tell which one. Both Phonetic and standard (BDS) are common
(16,16,54) atodorov: a safe bet is to go with phonetic I guess.
(16,19,46) rtcm: atodorov: can you tell me which one is it in XKB terms? is it "bg", "bg+bas_phonetic" or "bg+phonetic" ?
(16,20,51) rtcm: they're labeled as "Bulgarian", "Bulgarian (new phonetic)" and "Bulgarian (traditional phonetic)"
(16,21,30) atodorov: bg+phonetic is the traditional phonetic
(16,22,04) atodorov: bg labeled as "Bulgarian" is the standard one I guess. Here we call it BDS after the standardization institute
(16,22,34) atodorov: bg+bas_phonetic is created from the Bulgarian Academy of Science and is not very popular as far as I know. I've never seen it in use
(16,23,15) rtcm: atodorov: all I want to know is what most people would expect? like what does windows do by default? that's a good bet
(16,27,46) atodorov: rtcm: I'm just being told that new Windows releases use yet another layout by default, which is like phonetic but with some characters in new places and people don't like that
(16,27,53) atodorov: my safe bet goes to bg+phonetic
(16,29,05) rtcm: ok, thanks
It is a rare occasion when you get to make a decision that affects a large group of people and I hope you don't hate me for that!
Do you want to see more Fedora Test Days happening in Sofia? Join me and I will organize some more!
There are comments.
Here is a quick summary of the first Fedora Test Day in Sofia I hosted at init Lab today.
Attendance was quite poor, actually nobody else except me participated but almost nobody else visited the hackespace as well. I get that it is a working day and Test Days conflict with regular business hours but this is not going to change anyway. On the other hand where were all the freelancers and non-office job workers who usually hang around in the Lab? I have no idea!
On IRC there was much better activity, 5 or 6 people were testing across Asia, Europe and USA time zones. You can see the test results here. I've started filing quite a few bugs in the morning and continued well into the afternoon. I've managed to file a total of 10 bugs. Some of them were not related to virtualization and some of them turned out to be duplicates or not a bug. I even managed to file 2 duplicate bugs which likely have the same root cause myself :).
I've also experienced two bugs filed by other people: RHBZ #967371 for MATE desktop and RHBZ #1015636 for virt-manager's Save/Restore functionality.
I've tried ARM on x86_64 but that didn't get anywhere near a running system. I will make another post about ARM and what I've discovered there.
The one thing I liked is the test results application. It is not what I'm used to when dealing with RHEL, has far less features but is very fast and easy to use and suits the Test Days participants just fine. And is definitely much easier to use compared to filing results in the wiki.
Overall Fedora 20 virtualization status according to me is pretty good.
I hope to see more attendance on Thursday when we're going to test GNOME 3.10.
There are comments.
Fedora 20 Virtualization and GNOME test days will be tomorrow (8th Oct) and on Thursday (10th Oct)! Local community in Sofia will gather at init Lab! We start at 10:00 and everybody is welcome.
If you have no idea what I'm talking about check my previous posts and the announcement at init Lab's website:
See you there!
There are comments.
As I mentioned earlier I will organize some Fedora testing in Sofia. Here's an outline of my talk this Saturday and a general guide for anyone who wants to participate.
Fedora Test Days are day long events focused on testing a particular feature in the upcoming Fedora release. At the time of writing they are focused on Fedora 20.
Time and place for two test days will be announced at the end of the week and earlier next week so stay tuned!
There are comments.
On September 28th I will be giving a short talk about Fedora Test Days at the regular Linux for Bulgarians conference. I will explain what these are and how one can participate. I will also announce my plans and schedule to organize some Fedora 20 test days locally in Sofia. If you are a fan of Fedora and want to file bugs and kick some developers' ass this is the way to do it!
Other talks include Alexander Shopov's „Oracle's take on NoSQL“ which I wanted to hear since this summer and TBA talks about MicroTik routers.
The conference will take place on September 28th at the French Institute at Sofia University (see map). It starts at 13:00 and my talk should be at the beginning. See you there!
PS: this post was initially written on my BlackBerry Z10 with Penzus Editor - another small step in retiring my laptop.
There are comments.