Re: [ncc-services-wg] [db-wg] Urgent release of RIPE Database software
Dear Johan, thanks a lot for the new much more transparent mode of software version management for the RIPE data base service. This is a huge improvement. Nevertheless I think that we need continued discussion in the community to improve mutual understanding of considerations and impact. Your message today brings up few consideratiosn from me (a bit more tuned to the general pro cess than the specific case).
We were recently made aware of some issues with the documentation related to the REST API. To fix these issues requires a new release of the Database software as some parts of the documentation are generated from the active code. The documentation is vital for anyone wanting to use the service. We therefore believe it is necessary to make a new release containing this fix only and deploy it directly to production later today. The problem description does NOT read like there IS ACTUAL SERVICE IMPACT at the moment; looking at the RIPE NCC service status page seems to confirm since it reports "no known problems" and no ser vice announcements either.
Doing a software version change with less than a day advance notice certainly can be appropriate as an "emergency update" to deal with some service impact. How much harm is done if fix to the documentation is done a few days later while a note is posted indicating that certain errors will be fixed in the near future... ?
In this particular case we don=92t believe it is necessary to put the release in test beforehand. The only fix is for the documentation and this is preventing users form using the service, so it's quite urgent. This fix should have no impact on any other part of the RIPE Database service. I am tempted to believe your assessment; I dont have access to the changes and the resources to do my own full assessment. As I did experience 3 service impacting bugs within 12 months with UNannounced version changes I conclude that I rather see than believe. I certainly do see a need to watch more carefully how software behaves after ANY change and a non zero probability of some surprise (that could result in service impact for me).
Note: knowing the schedule of upcoming changes some time in advance can be more important than actually having the ability to run tests (like due to IETF timing I could not run tests against the test server with 1.67.4 - but we did schedule preparation of critical production runs in a way that protected us against potential surprises due to the software change.)
If you have any questions then please let us know.
Kind regards,
Johan =C5hl=E9n Assistant Manager Database RIPE NCC= Best regards, Ruediger
Ruediger Volk Deutsche Telekom AG -- Internet Backbone Engineering E-Mail: rv@NIC.DTAG.DE
Dear Ruediger, Thank you for the input. As you are all aware the release procedure is new and does require some tuning, so in that aspect any suggestions on how we can improve this are welcome. In this particular situation our assessment is that the broken links in the documentation indeed has a service impact. For some users this could be critical as they may not be able to use the REST API service. At least one case was reported to us where the development towards the API has come to a halt because of this. Perhaps in this case we can hold the deployment of the fix until the working group advices us on how to proceed. Kind regards, Johan Åhlén Assistant Database Manager RIPE NCC On 15 Aug 2013, at 14:52, "Ruediger Volk, Deutsche Telekom Technik - FMED-41.." <rv@NIC.DTAG.DE> wrote:
Dear Johan,
thanks a lot for the new much more transparent mode of software version management for the RIPE data base service. This is a huge improvement.
Nevertheless I think that we need continued discussion in the community to improve mutual understanding of considerations and impact.
Your message today brings up few consideratiosn from me (a bit more tuned to the general pro cess than the specific case).
We were recently made aware of some issues with the documentation related to the REST API. To fix these issues requires a new release of the Database software as some parts of the documentation are generated from the active code. The documentation is vital for anyone wanting to use the service. We therefore believe it is necessary to make a new release containing this fix only and deploy it directly to production later today. The problem description does NOT read like there IS ACTUAL SERVICE IMPACT at the moment; looking at the RIPE NCC service status page seems to confirm since it reports "no known problems" and no ser vice announcements either.
Doing a software version change with less than a day advance notice certainly can be appropriate as an "emergency update" to deal with some service impact. How much harm is done if fix to the documentation is done a few days later while a note is posted indicating that certain errors will be fixed in the near future... ?
In this particular case we don=92t believe it is necessary to put the release in test beforehand. The only fix is for the documentation and this is preventing users form using the service, so it's quite urgent. This fix should have no impact on any other part of the RIPE Database service. I am tempted to believe your assessment; I dont have access to the changes and the resources to do my own full assessment. As I did experience 3 service impacting bugs within 12 months with UNannounced version changes I conclude that I rather see than believe. I certainly do see a need to watch more carefully how software behaves after ANY change and a non zero probability of some surprise (that could result in service impact for me).
Note: knowing the schedule of upcoming changes some time in advance can be more important than actually having the ability to run tests (like due to IETF timing I could not run tests against the test server with 1.67.4 - but we did schedule preparation of critical production runs in a way that protected us against potential surprises due to the software change.)
If you have any questions then please let us know.
Kind regards,
Johan =C5hl=E9n Assistant Manager Database RIPE NCC= Best regards, Ruediger
Ruediger Volk
Deutsche Telekom AG -- Internet Backbone Engineering
E-Mail: rv@NIC.DTAG.DE
Hi, Any reason not to deploy on test environmen, wait 24 hours, proceed to roll out on production environment? Kind regards, Job On Aug 15, 2013, at 3:27 PM, Johan Åhlén <jahlen@ripe.net> wrote:
Dear Ruediger,
Thank you for the input. As you are all aware the release procedure is new and does require some tuning, so in that aspect any suggestions on how we can improve this are welcome.
In this particular situation our assessment is that the broken links in the documentation indeed has a service impact. For some users this could be critical as they may not be able to use the REST API service. At least one case was reported to us where the development towards the API has come to a halt because of this.
Perhaps in this case we can hold the deployment of the fix until the working group advices us on how to proceed.
Kind regards,
Johan Åhlén Assistant Database Manager RIPE NCC
On 15 Aug 2013, at 14:52, "Ruediger Volk, Deutsche Telekom Technik - FMED-41.." <rv@NIC.DTAG.DE> wrote:
Dear Johan,
thanks a lot for the new much more transparent mode of software version management for the RIPE data base service. This is a huge improvement.
Nevertheless I think that we need continued discussion in the community to improve mutual understanding of considerations and impact.
Your message today brings up few consideratiosn from me (a bit more tuned to the general pro cess than the specific case).
We were recently made aware of some issues with the documentation related to the REST API. To fix these issues requires a new release of the Database software as some parts of the documentation are generated from the active code. The documentation is vital for anyone wanting to use the service. We therefore believe it is necessary to make a new release containing this fix only and deploy it directly to production later today. The problem description does NOT read like there IS ACTUAL SERVICE IMPACT at the moment; looking at the RIPE NCC service status page seems to confirm since it reports "no known problems" and no ser vice announcements either.
Doing a software version change with less than a day advance notice certainly can be appropriate as an "emergency update" to deal with some service impact. How much harm is done if fix to the documentation is done a few days later while a note is posted indicating that certain errors will be fixed in the near future... ?
In this particular case we don=92t believe it is necessary to put the release in test beforehand. The only fix is for the documentation and this is preventing users form using the service, so it's quite urgent. This fix should have no impact on any other part of the RIPE Database service. I am tempted to believe your assessment; I dont have access to the changes and the resources to do my own full assessment. As I did experience 3 service impacting bugs within 12 months with UNannounced version changes I conclude that I rather see than believe. I certainly do see a need to watch more carefully how software behaves after ANY change and a non zero probability of some surprise (that could result in service impact for me).
Note: knowing the schedule of upcoming changes some time in advance can be more important than actually having the ability to run tests (like due to IETF timing I could not run tests against the test server with 1.67.4 - but we did schedule preparation of critical production runs in a way that protected us against potential surprises due to the software change.)
If you have any questions then please let us know.
Kind regards,
Johan =C5hl=E9n Assistant Manager Database RIPE NCC= Best regards, Ruediger
Ruediger Volk
Deutsche Telekom AG -- Internet Backbone Engineering
E-Mail: rv@NIC.DTAG.DE
-- AS5580 - Atrato IP Networks
That seems like a reasonable approach to me. Tim Garrison Software Engineer III *SoftLayer, an IBM Company* 315 Capitol Street Suite 205, Houston, TX 77002 281.714.4213 direct | 713.540.4325 mobile | 281.714.4657 fax | tgarrison@softlayer.com On 08/15/2013 08:32 AM, Job Snijders wrote:
Hi,
Any reason not to deploy on test environmen, wait 24 hours, proceed to roll out on production environment?
Kind regards,
Job
On Aug 15, 2013, at 3:27 PM, Johan Åhlén <jahlen@ripe.net> wrote:
Dear Ruediger,
Thank you for the input. As you are all aware the release procedure is new and does require some tuning, so in that aspect any suggestions on how we can improve this are welcome.
In this particular situation our assessment is that the broken links in the documentation indeed has a service impact. For some users this could be critical as they may not be able to use the REST API service. At least one case was reported to us where the development towards the API has come to a halt because of this.
Perhaps in this case we can hold the deployment of the fix until the working group advices us on how to proceed.
Kind regards,
Johan Åhlén Assistant Database Manager RIPE NCC
On 15 Aug 2013, at 14:52, "Ruediger Volk, Deutsche Telekom Technik - FMED-41.." <rv@NIC.DTAG.DE> wrote:
Dear Johan,
thanks a lot for the new much more transparent mode of software version management for the RIPE data base service. This is a huge improvement.
Nevertheless I think that we need continued discussion in the community to improve mutual understanding of considerations and impact.
Your message today brings up few consideratiosn from me (a bit more tuned to the general pro cess than the specific case).
We were recently made aware of some issues with the documentation related to the REST API. To fix these issues requires a new release of the Database software as some parts of the documentation are generated from the active code. The documentation is vital for anyone wanting to use the service. We therefore believe it is necessary to make a new release containing this fix only and deploy it directly to production later today. The problem description does NOT read like there IS ACTUAL SERVICE IMPACT at the moment; looking at the RIPE NCC service status page seems to confirm since it reports "no known problems" and no ser vice announcements either.
Doing a software version change with less than a day advance notice certainly can be appropriate as an "emergency update" to deal with some service impact. How much harm is done if fix to the documentation is done a few days later while a note is posted indicating that certain errors will be fixed in the near future... ?
In this particular case we don=92t believe it is necessary to put the release in test beforehand. The only fix is for the documentation and this is preventing users form using the service, so it's quite urgent. This fix should have no impact on any other part of the RIPE Database service. I am tempted to believe your assessment; I dont have access to the changes and the resources to do my own full assessment. As I did experience 3 service impacting bugs within 12 months with UNannounced version changes I conclude that I rather see than believe. I certainly do see a need to watch more carefully how software behaves after ANY change and a non zero probability of some surprise (that could result in service impact for me).
Note: knowing the schedule of upcoming changes some time in advance can be more important than actually having the ability to run tests (like due to IETF timing I could not run tests against the test server with 1.67.4 - but we did schedule preparation of critical production runs in a way that protected us against potential surprises due to the software change.)
If you have any questions then please let us know.
Kind regards,
Johan =C5hl=E9n Assistant Manager Database RIPE NCC= Best regards, Ruediger
Ruediger Volk
Deutsche Telekom AG -- Internet Backbone Engineering
E-Mail: rv@NIC.DTAG.DE
Dear all, The release containing the fix is now available in the TEST Database and there's a service announcement on the web page. If there are no objections to this we will proceed with 24h in the TEST Database and put this release in production on the RIPE Database tomorrow at 16:00. Please go ahead and test your applications against the version in the TEST Database. The URL for the REST API in TEST is: http(s)://rest-test.db.ripe.net Kind regards, Johan Åhlén Assistant Manager Database RIPE NCC On 15 Aug 2013, at 15:37, Tim Garrison <tgarrison@softlayer.com> wrote:
That seems like a reasonable approach to me.
Tim Garrison Software Engineer III
SoftLayer, an IBM Company 315 Capitol Street Suite 205, Houston, TX 77002 281.714.4213 direct | 713.540.4325 mobile | 281.714.4657 fax | tgarrison@softlayer.com
On 08/15/2013 08:32 AM, Job Snijders wrote:
Hi,
Any reason not to deploy on test environmen, wait 24 hours, proceed to roll out on production environment?
Kind regards,
Job
On Aug 15, 2013, at 3:27 PM, Johan Åhlén <jahlen@ripe.net> wrote:
Dear Ruediger,
Thank you for the input. As you are all aware the release procedure is new and does require some tuning, so in that aspect any suggestions on how we can improve this are welcome.
In this particular situation our assessment is that the broken links in the documentation indeed has a service impact. For some users this could be critical as they may not be able to use the REST API service. At least one case was reported to us where the development towards the API has come to a halt because of this.
Perhaps in this case we can hold the deployment of the fix until the working group advices us on how to proceed.
Kind regards,
Johan Åhlén Assistant Database Manager RIPE NCC
On 15 Aug 2013, at 14:52, "Ruediger Volk, Deutsche Telekom Technik - FMED-41.." <rv@NIC.DTAG.DE> wrote:
Dear Johan,
thanks a lot for the new much more transparent mode of software version management for the RIPE data base service. This is a huge improvement.
Nevertheless I think that we need continued discussion in the community to improve mutual understanding of considerations and impact.
Your message today brings up few consideratiosn from me (a bit more tuned to the general pro cess than the specific case).
We were recently made aware of some issues with the documentation related to the REST API. To fix these issues requires a new release of the Database software as some parts of the documentation are generated from the active code. The documentation is vital for anyone wanting to use the service. We therefore believe it is necessary to make a new release containing this fix only and deploy it directly to production later today. The problem description does NOT read like there IS ACTUAL SERVICE IMPACT at the moment; looking at the RIPE NCC service status page seems to confirm since it reports "no known problems" and no ser vice announcements either.
Doing a software version change with less than a day advance notice certainly can be appropriate as an "emergency update" to deal with some service impact. How much harm is done if fix to the documentation is done a few days later while a note is posted indicating that certain errors will be fixed in the near future... ?
In this particular case we don=92t believe it is necessary to put the release in test beforehand. The only fix is for the documentation and this is preventing users form using the service, so it's quite urgent. This fix should have no impact on any other part of the RIPE Database service. I am tempted to believe your assessment; I dont have access to the changes and the resources to do my own full assessment. As I did experience 3 service impacting bugs within 12 months with UNannounced version changes I conclude that I rather see than believe. I certainly do see a need to watch more carefully how software behaves after ANY change and a non zero probability of some surprise (that could result in service impact for me).
Note: knowing the schedule of upcoming changes some time in advance can be more important than actually having the ability to run tests (like due to IETF timing I could not run tests against the test server with 1.67.4 - but we did schedule preparation of critical production runs in a way that protected us against potential surprises due to the software change.)
If you have any questions then please let us know.
Kind regards,
Johan =C5hl=E9n Assistant Manager Database RIPE NCC= Best regards, Ruediger
Ruediger Volk
Deutsche Telekom AG -- Internet Backbone Engineering
E-Mail: rv@NIC.DTAG.DE
I would like to report that there appear to be some broken links in the documentation still. For example, on https://rest-test.db.ripe.net/api-doc/path__create.html there is a link to the "whois-resources" element, which points to https://rest-test.db.ripe.net/api-doc/el_ns0_whois-resources.html . This link produces a 404. Tim Garrison Software Engineer III *SoftLayer, an IBM Company* 315 Capitol Street Suite 205, Houston, TX 77002 281.714.4213 direct | 713.540.4325 mobile | 281.714.4657 fax | tgarrison@softlayer.com On 08/15/2013 09:10 AM, Johan Åhlén wrote:
Dear all,
The release containing the fix is now available in the TEST Database and there's a service announcement on the web page.
If there are no objections to this we will proceed with 24h in the TEST Database and put this release in production on the RIPE Database tomorrow at 16:00. Please go ahead and test your applications against the version in the TEST Database.
The URL for the REST API in TEST is:
http(s)://rest-test.db.ripe.net <http://rest-test.db.ripe.net>
Kind regards,
Johan Åhlén Assistant Manager Database RIPE NCC
On 15 Aug 2013, at 15:37, Tim Garrison <tgarrison@softlayer.com <mailto:tgarrison@softlayer.com>> wrote:
That seems like a reasonable approach to me.
Tim Garrison Software Engineer III
*SoftLayer, an IBM Company* 315 Capitol Street Suite 205, Houston, TX 77002 281.714.4213 direct | 713.540.4325 mobile | 281.714.4657 fax | tgarrison@softlayer.com
On 08/15/2013 08:32 AM, Job Snijders wrote:
Hi,
Any reason not to deploy on test environmen, wait 24 hours, proceed to roll out on production environment?
Kind regards,
Job
On Aug 15, 2013, at 3:27 PM, Johan Åhlén<jahlen@ripe.net> wrote:
Dear Ruediger,
Thank you for the input. As you are all aware the release procedure is new and does require some tuning, so in that aspect any suggestions on how we can improve this are welcome.
In this particular situation our assessment is that the broken links in the documentation indeed has a service impact. For some users this could be critical as they may not be able to use the REST API service. At least one case was reported to us where the development towards the API has come to a halt because of this.
Perhaps in this case we can hold the deployment of the fix until the working group advices us on how to proceed.
Kind regards,
Johan Åhlén Assistant Database Manager RIPE NCC
On 15 Aug 2013, at 14:52, "Ruediger Volk, Deutsche Telekom Technik - FMED-41.."<rv@NIC.DTAG.DE> wrote:
Dear Johan,
thanks a lot for the new much more transparent mode of software version management for the RIPE data base service. This is a huge improvement.
Nevertheless I think that we need continued discussion in the community to improve mutual understanding of considerations and impact.
Your message today brings up few consideratiosn from me (a bit more tuned to the general pro cess than the specific case).
We were recently made aware of some issues with the documentation related to the REST API. To fix these issues requires a new release of the Database software as some parts of the documentation are generated from the active code. The documentation is vital for anyone wanting to use the service. We therefore believe it is necessary to make a new release containing this fix only and deploy it directly to production later today. The problem description does NOT read like there IS ACTUAL SERVICE IMPACT at the moment; looking at the RIPE NCC service status page seems to confirm since it reports "no known problems" and no ser vice announcements either.
Doing a software version change with less than a day advance notice certainly can be appropriate as an "emergency update" to deal with some service impact. How much harm is done if fix to the documentation is done a few days later while a note is posted indicating that certain errors will be fixed in the near future... ?
In this particular case we don=92t believe it is necessary to put the release in test beforehand. The only fix is for the documentation and this is preventing users form using the service, so it's quite urgent. This fix should have no impact on any other part of the RIPE Database service. I am tempted to believe your assessment; I dont have access to the changes and the resources to do my own full assessment. As I did experience 3 service impacting bugs within 12 months with UNannounced version changes I conclude that I rather see than believe. I certainly do see a need to watch more carefully how software behaves after ANY change and a non zero probability of some surprise (that could result in service impact for me).
Note: knowing the schedule of upcoming changes some time in advance can be more important than actually having the ability to run tests (like due to IETF timing I could not run tests against the test server with 1.67.4 - but we did schedule preparation of critical production runs in a way that protected us against potential surprises due to the software change.)
If you have any questions then please let us know.
Kind regards,
Johan =C5hl=E9n Assistant Manager Database RIPE NCC= Best regards, Ruediger
Ruediger Volk
Deutsche Telekom AG -- Internet Backbone Engineering
E-Mail:rv@NIC.DTAG.DE
Dear Tim, Instead of quickly fixing this issue and creating a new release today we've decided to halt the deployment of any new release for now. This is very embarrassing and we're very sorry for this. We obviously put too much trust in the framework that generates the documentation and we don't have the proper mechanisms in place to validate the correctness of the documentation generated. We will give this our fullest attention the coming days and expect to have a working solution sometime next week. In the meantime I hope the existing documentation in TEST Database contains the information you need to proceed with your project. If you have any questions about how the API works then please contact us directly and we'll gladly help you out. Kind regards, Johan Åhlén Assistant Manager Database RIPE NCC On 15 Aug 2013, at 17:20, Tim Garrison <tgarrison@softlayer.com> wrote:
I would like to report that there appear to be some broken links in the documentation still. For example, on https://rest-test.db.ripe.net/api-doc/path__create.html there is a link to the "whois-resources" element, which points to https://rest-test.db.ripe.net/api-doc/el_ns0_whois-resources.html . This link produces a 404.
Tim Garrison Software Engineer III
SoftLayer, an IBM Company 315 Capitol Street Suite 205, Houston, TX 77002 281.714.4213 direct | 713.540.4325 mobile | 281.714.4657 fax | tgarrison@softlayer.com
On 08/15/2013 09:10 AM, Johan Åhlén wrote:
Dear all,
The release containing the fix is now available in the TEST Database and there's a service announcement on the web page.
If there are no objections to this we will proceed with 24h in the TEST Database and put this release in production on the RIPE Database tomorrow at 16:00. Please go ahead and test your applications against the version in the TEST Database.
The URL for the REST API in TEST is:
http(s)://rest-test.db.ripe.net
Kind regards,
Johan Åhlén Assistant Manager Database RIPE NCC
On 15 Aug 2013, at 15:37, Tim Garrison <tgarrison@softlayer.com> wrote:
That seems like a reasonable approach to me.
Tim Garrison Software Engineer III
SoftLayer, an IBM Company 315 Capitol Street Suite 205, Houston, TX 77002 281.714.4213 direct | 713.540.4325 mobile | 281.714.4657 fax | tgarrison@softlayer.com
On 08/15/2013 08:32 AM, Job Snijders wrote:
Hi,
Any reason not to deploy on test environmen, wait 24 hours, proceed to roll out on production environment?
Kind regards,
Job
On Aug 15, 2013, at 3:27 PM, Johan Åhlén <jahlen@ripe.net> wrote:
Dear Ruediger,
Thank you for the input. As you are all aware the release procedure is new and does require some tuning, so in that aspect any suggestions on how we can improve this are welcome.
In this particular situation our assessment is that the broken links in the documentation indeed has a service impact. For some users this could be critical as they may not be able to use the REST API service. At least one case was reported to us where the development towards the API has come to a halt because of this.
Perhaps in this case we can hold the deployment of the fix until the working group advices us on how to proceed.
Kind regards,
Johan Åhlén Assistant Database Manager RIPE NCC
On 15 Aug 2013, at 14:52, "Ruediger Volk, Deutsche Telekom Technik - FMED-41.." <rv@NIC.DTAG.DE> wrote:
Dear Johan,
thanks a lot for the new much more transparent mode of software version management for the RIPE data base service. This is a huge improvement.
Nevertheless I think that we need continued discussion in the community to improve mutual understanding of considerations and impact.
Your message today brings up few consideratiosn from me (a bit more tuned to the general pro cess than the specific case).
> We were recently made aware of some issues with the documentation > related to the REST API. To fix these issues requires a new release of > the Database software as some parts of the documentation are generated > from the active code. The documentation is vital for anyone wanting to > use the service. We therefore believe it is necessary to make a new > release containing this fix only and deploy it directly to production > later today. The problem description does NOT read like there IS ACTUAL SERVICE IMPACT at the moment; looking at the RIPE NCC service status page seems to confirm since it reports "no known problems" and no ser vice announcements either.
Doing a software version change with less than a day advance notice certainly can be appropriate as an "emergency update" to deal with some service impact. How much harm is done if fix to the documentation is done a few days later while a note is posted indicating that certain errors will be fixed in the near future... ?
> In this particular case we don=92t believe it is necessary to put the > release in test beforehand. The only fix is for the documentation and > this is preventing users form using the service, so it's quite urgent. > This fix should have no impact on any other part of the RIPE Database > service. I am tempted to believe your assessment; I dont have access to the changes and the resources to do my own full assessment. As I did experience 3 service impacting bugs within 12 months with UNannounced version changes I conclude that I rather see than believe. I certainly do see a need to watch more carefully how software behaves after ANY change and a non zero probability of some surprise (that could result in service impact for me).
Note: knowing the schedule of upcoming changes some time in advance can be more important than actually having the ability to run tests (like due to IETF timing I could not run tests against the test server with 1.67.4 - but we did schedule preparation of critical production runs in a way that protected us against potential surprises due to the software change.)
> If you have any questions then please let us know. > > Kind regards, > > Johan =C5hl=E9n > Assistant Manager Database > RIPE NCC= Best regards, Ruediger
Ruediger Volk
Deutsche Telekom AG -- Internet Backbone Engineering
E-Mail: rv@NIC.DTAG.DE
Hi,
Any reason not to deploy on test environmen, wait 24 hours, proceed to roll out on production environment? would seem to me: deploying the fix to the test environment would already solve the service impact (that Johan sees) ... if we can point potential readers of the documentation there... So sidestepping the test environment actually delays fixing the specific
Hi Job, Johan, I'm happy to see careful attention, improvements, and tuning for the new process. That needs to continue and I'd expect to have some discussion in Athens... problem - while increasing risks for the production use... My take: no need and justification for rushing change on the production system. Kind regards, Ruediger Ruediger Volk Deutsche Telekom AG -- Internet Backbone Engineering E-Mail: rv@NIC.DTAG.DE
participants (5)
-
Job Snijders -
Johan Åhlén -
Ruediger Volk, Deutsche Telekom Technik - FMED-41.. -
Ruediger Volk, Deutsche Telekom Technik - FMED-41..
-
Tim Garrison