Planet MVS User Experiences OS/390 R6 upgrade experiences

Installation: ServerPac

• You spend a couple of hours rearranging the logical volumes to match the physical volumes. Too bad they don't change the install have two options:
  • Masochist - The current method: you do all the work moving datasets around.
  • Non-Masochist - The datasets come configured how most people (I would guess) would want for the Full Install on 3390-3 devices:
    • Master Catalog and SMP volume
    • SYSRES-1 (traditional stuff that was always on SYSRES)
    • SYSRES-2 (New stuff past MVS v5 and things you aren't licensed for at your shop)
    • DLIB-1
    • DLIB-2
    • SMS/HFS stuff
  See more on my OS/390 install volume layout on my MVS Tips page.
  
  
• The job that installs the ServerPac dialog files (if I remember right) creates JCL with a long line that endes with "LABEL=(xx,SL)" with no room to add ",,,EXPDT=98000". You get to use the ISPF commands: X ALL;FIND LABEL= ALL and then the "TS" line command on all of the edit lines, add some // on each and then do the change all for 98000 string. Whew!
  
  
• Unless I miss it each time, IBM doesn't give you "TMS=YES" option do add the EXPDT=98000 (foreign tape) to the JCL. I seem to remember that back in my MVS 4.3 install, that there was some type of "Rexx exit" that allowed me to customize the JCL being built but I could be remembering that wrong.
  
  
• Since I already have a HLQ of SMPE and forgot to fix this upfront, I butchered the system midstream in the ServerPac install to add a HLQ of SMPER6 so I could keep my SMPE HLQ for R2. All of my future installs will have the release in the SMPE HLQ. I read or heard this tip somewhere and it makes perfect sense.

Installation: Program Products

The hardest part of an OS/390 upgrade is getting all of the program products upgraded to a compatible level.

TCP/IP

PARMLIB concatenation

*        |<------------- Dataset name ------------->| Volume
*---+----1----+----2----+----3----+----4----+----5----+----6----+----
PARMLIB  SYS1.PARMLIB.OS390R2.CSW                     *MCAT*
PARMLIB  SYS1.PARMLIB                                 *MCAT*
PARMLIB  SYS1.PARMLIB.OS390R2.IBM                     *MCAT*

PARMLIB Concatenation - HowIShotMyselfInTheFoot section

• SYS1.PROCLIB(BLSJPRMI) was at the MVS V4 level and had a IPCSPARM DD statement pointing directly to SYS1.PARMLIB (which after my new PARMLIB concatenation process had none of the IPCS members). The latest/greatest has this statement removed to support PARMLIB concatenations.
  
  FYI: BLSJPRMI initializes for SNAP (SYSUDUMPs too) processing.

PROCLIB concatenation

//MSTJCL10 JOB MSGLEVEL=(1,1),TIME=1440
//         EXEC PGM=IEEMB860,DPRTY=(15,15)
//STCINRDR DD SYSOUT=(A,INTRDR)
//TSOINRDR DD SYSOUT=(A,INTRDR)
//IEFPDSI  DD DSN=SYS1.PROCLIB.OS390R2.CSW,DISP=SHR
//         DD DSN=SYS1.PROCLIB,DISP=SHR
//         DD DSN=SYS1.PROCLIB.OS390R2.IBM,DISP=SHR
//IEFJOBS  DD DSN=SYSL.SYS.STCJOBS.CSW1MVS,DISP=SHR
//SYSUADS  DD DSN=SYS1.UADS,DISP=SHR
//SYSLBC   DD DSN=SYS1.BRODCAST,DISP=SHR
/*

//JES2    PROC ENTRY=HASJES20,
//        P='WARM,NOREQ',
...
//************************************
//IEFPROC  EXEC PGM=&ENTRY.,TIME=1440,
//         DPRTY=(15,15),PARM='&P'
//HASPLIST DD DDNAME=IEFRDER
//*
//PROC00   DD DISP=SHR,DSN=SYS1.PROCLIB.OS390R2.CSW
//         DD DISP=SHR,DSN=SYS1.PROCLIB
//         DD DISP=SHR,DSN=SYS1.PROCLIB.OS390R2.IBM
//         DD DISP=SHR,DSN=CEE.SCEEPROC
//         DD DISP=SHR,DSN=user.proclibs here
...

CS/390 pre-installation

TCP/IP

VTAM

FIXED MAX(120M)
ECSA  MAX(120M)

Useful Mailing-lists and Newsgroups

• Mailing lists:
  • IBMTCP-L mailing list - TCP/IP for MVS (OS/390), VM, VSE

    Join the IBMTCP_-L mailing list by by sending an email with "sub IBMTCP-L your name" in the body to listserv@vm.marist.edu

  • MVS OpenEdition mailing list - OpenMVS/OpenEdition/Unix_System_Services
    (subscription form available at bottom of this page)
• Non-IBM served Newsgroup:
  • IBM-Main - bit.listserv.ibm-main
• IBM served newsgroups:
  • CS/390 (TCP/IP) - ibm.software.commserver.os390.ip
  • CS/390 (VTAM) - ibm.software.commserver.os390.sna

Problems found in production

Clarification: Most of these errors were found in production because we lost two scheduled test times due to customer needs. We knew going in that we would be finding a few things to correct. We had one prior short test in the production environment.

• TCP/IP
  • In TCP/IP 3.5, I took advantage of the new USS MSG 0-13 (MSG10 was already supported) which broke our screen scraping scripts.

    Solution: USS messages other than MSG10 were removed.

  • 17 May 1999 - Up until TCP/IP 3.5, IBM passed data past the application keyword to the VTAM application. For example, you could type in "TPX userid password" and "userid" and "password" were passed to the TPX application. In R6 we found that TCP/IP's USS table works more like VTAM's USS table and doesn't pass application data by default.

    Solution:Currently using the USS implementation ASIS without adding data passing.

  • When I changed the USS screen dynamically, I used the "V TCPIP,tcpip,O,obeyfile" operator command. This successfully changed the USS screen okay but then all of our AUTOLOG tasks started being stopped and started by TCP/IP every 5 minutes. There were no hits for the search arguments I used on IBMLINK for this problem.

    Solution: Waited for next bounce of TCP/IP to clear problem. The operators had to manually clean the SPOOL of canceled TCPIP AUTOLOG tasks.

  • The CICS systems programmer reported that the CICS sockets interface got a no processes available error. I dynamically bumped up this parameter via the operator command "SETOMVS MAXPROCSYS=4000". A little bit later another sysprog comes in and warns me that a vendor (IBI) told him that if you set MAXPROCSYS too high (say 4000) then TCP/IP goes haywire. About this time we noticed that that TCP/IP had gone haywire and we got a user friendly wait state (which we attributed to that).

    Solution: Set MAXPROCSYS to 2000

  • Oops, I forgot to put our second TCP/IP stack in SYS1.PARMLIB(BPXPRMxx)' and I also committed the sin of using INET instead of CINET. I didn't see how I could change this dynamically.

    Solution: IPL to add a second TCP/IP task!!! How User-Friendly!

    24 May 1999 - Looking back I now see why I used INET: The ServerPac comes with a manual called "ServerPac: Installing Your Order". Mine said: BPXPRMxx member should be updated to contain the following: FILESYSTYPE TYPE(INET) ENTRYPOINT(EZBPFINI) SUBFILESYSTYPE NAME(TCPIP) TYPE(INET) ENTRYPOINT(EZBPFINI) NETWORK DOMAINNAME(AF_INET) DOMAINNUMBER(2) MAXSOCKETS(10000) TYPE(INET) This is good advice AS LONG AS YOU ONLY NEED TO RUN ONE TCP/IP STACK.

    
    
    
  • I used the IBM sample TCP/IP profile for a template which had

                TELNETPARMS
                   INACTIVE 600
                ENDTELNETPARMS
                

    This caused TN3270 users to time out after 10 minutes of inactivity (Not good!). Trying to change this value dynamically via the operator command "VARY TCPIP,TCPIP,O,obey-file-name" did not change the value. (which you can check via "D TCPIP,TCPIP,T,PROF").
    
    I opened an ETR on IBMLINK about this and they responded that for options in TELNETPARMS ... ENDTELNETPARMS to be changed you must also code the BEGINVTAM ... ENDVTAM in the Obeyfile as well.

    Solution: It works when you do it right!

  • This may also happen in previous releases of IBM's TCP/IP for MVS but for this release I went to using sequential datasets for the //PROFILE (TCPIP.PROFILE.TCPIP) and //SYSTCPD (TCPIP.TCPIP.DATA). Even though they are allocated with DISP=SHR in the TCPIP started task, TCP/IP enqueued on them so that you couldn't edit them via ISPF. This also means that you can't easily pre-stage future changes to TCP/IP.

    Solution: I went back to using members in SYS1.TCPPARMS.
    
    John M Krogulecki emailed me that:

    If you stick a free=close on them, you can edit them with the STC running.
    Our proc, for instance, has:

    //PROFILE  DD DISP=SHR,DSN=TCPIP.&SYSNAME..TCPIP,FREE=CLOSE
    

  • When we try to shutdown OS/390 R6, we get to the point where we think it's time to do the "$PJES2" command. A "$DU,ACT" shows nothing active but JES2 says he isn't dormant.

    Solution: Barry Dent sent me an email:

    APAR OW29618 explains the JES2 shutdown problem, you need to issue

         F BPXOINIT,SHUTDOWN=FORKINIT
    

    to cancel any BPXAS address spaces.

    I'd also like to thank Peter Briedis and Matt Martin who also emailed me with this fix.

• VTAM
  • Actually found on our test machine but I mention it here because you always want VTAM to start: You need need a //SISTCLIB in the VTAM proc JCL and it better be APF authorized!
    
    
  • Somewhere between R2 and R6, some defaults were changed and our VTAM guys had a fun time with these parameters which resulted in a couple of unscheduled IPLs:
    • Added XNETALS=YES to ATCSTRxx. This makes VTAM look like the previous versions way of checking XIDs.
    • ENCR=NONE had to code for LU62 connections to IDMS
    • DYNPU=YES had to code on NCP GROUP to get Dynamic PU's
  
  
  
• Non-CS/390 (TCP/IP and VTAM) IBM problems: None so far!

  17 May 1999 - We did find that IBM went back to enforcing the rule that you can't do a IDCAMS DELETE/DEFINE/REPRO-OUTFILE of the same dataset in the same step. If the IDCAMS step is changed to two steps or the OUTDATASET() is used instead of OUTFILE(), then it works. A few production jobs had to be modified. There is an APAR on this (sorry I don't have the number at home) dated 1987 and last modified in 1998. My S.W.A.G. is that this got changed at DFSMS 1.4.

ISV problems

• Sterling's VAM (SAMS:Allocate) needed PTF SS04257. Their hooks in the system backed out successfully on the test machine but not in production (of course).

  Solution: Unscheduled IPL

• SQLHOST (Peoplesoft/Centura) works for a while and then needs to be bounced.

  Solution: I made a post to the IBMTCP-L mailing list describing this problem and Joe.Kimberly posted back that Centura wrote them a fix for this problem. We then contacted Peoplesoft who then sent us a fix which has cleared up the problem.

• CA-Netspy caused TCP/IP to take a SVC dump and CA-Netspy terminated.

  Solution: 17 May 1999 - Applied fix from CA (sorry don't have PTF number)

What others have found

Dave,

Just read you r2-r6 upgrade - 2 minutes ago i just got caught out by the
INACTIVE 600 in telnetparms so thanks for the tip about BEGINVATM....

Another one we just hit though that hurt us & you might like to add to your
page.
In FTP.DATA under v1.2, 'RETPD 0' meant no expirey date but under v2.6 it
means exired immediately - very nasty - HSM's just been deleting loads of
stuff we didn't want it to! Fixed by romoving the '0' or RETPD completely.

Rgds, Jon.

Good News (so far)

• My JES2 exits (2,4,6,13,20,36) only needed to be reassembled and didn't have any assembly or execution errors.
  
  Although I *thought* that I heard on the IBM-MAIN newsgroup (or Share, or the OS/390 Expo, or the guy at 7-11) that Exit 6 would have to be rewritten, ours is running fine. Maybe what happened was that back in 1992, I used the IBM XINTKEY subroutine to search for text units - this may have shielded me from whatever (real or imagined) was removed or changed in R3, R4, R5 or R6.
  
  Will a $ACTIVATE get me?
  
  
• 24 May 1999 - I didn't mention this before: it's really nice to upgrade JES2 without a cold start!!!
  
  
• My SMF exits continued to run asis (no re-assembly required).
  
  
• Other program products seem to holding up fine.
  
  
• Management is very happy with the upgrade. We have a good solid current Operating System to go into the Year 2000.
  
  
• 9 June 1999 - We've been up continuously for a full month now (since 9 May 1999). Knock on Wood!. I haven't had to put on a single PTF past the recommended maintenance APARs (see above).

Advice

• In hindsight, I wish I would have saved output from the of "NETSTAT ALL", "NETSTAT SOCKETS", etc. commands from TCP/IP for MVS version 3.2 under OS/390 R2 for comparison after the upgrade to OS/390 R6.
  
  
• Call your ISV vendors to double check O/S compatibility at regular intervals. I found out about a couple of PTFs needed for CA90 Services the day *before* production. All turned out well...
  
  
• 24 May 1999 - You should also watch out for the Hyperdrive! It always seems to go out just when you need it. "Flying through hyperspace ain't like dusting crops, boy". Wait a second, that's not a problem with R6 after all.

  A long time ago in a MVS upgrade far far away...

• The last bit of advice is: Double check anything I put on this page for accuracy and how it applies to your shop. Your Mileage May Vary.

Corrections

• MAXSYSPROC should be MAXPROCSYS Changed(23:30 12-MAY-1999) - Thanks to Tina Morris and Richard Smith
• //ISTCLIB should be //SISTCLIB (VTAM started task) Changed(23:30 12-MAY-1999) - Thanks to Richard Smith