INTRODUCTION
py_enscribe is a python package developed to perform Enscribe File operations. It was released in SPR T0993L01^AAK. The design of this package was influenced by JEnscribe (Enscribe operations for Java). Internally py_enscribe uses cython to call Guardian APIs to achieve the Enscribe related operations. It has limited support for unstructured files and works well with other types of Enscribe Files.
GETTING STARTED
py_enscribe has different modules to achieve the desired functionalities. The modules have classes in them that have the same name as the modules themselves. Let us look at some of these, EnscribeFile module has EnscribeFile which provides access to Enscribe files through methods that perform standard Guardian operations.
EnscribeFileAttributes provides access to the more static information (attributes) about any Enscribe file. This information is obtained by a call to the Guardian procedure FILE_GETINFOLISTBYNAME_. There is no requirement that the file be open to use EnscribeFileAttributes. EnscribeFileAttributes also provides methods to change attributes when you want to alter the characteristics of an existing Enscribe file. These methods are comparable to the parameters of FILE_ALTERLIST_.
The AlternateKeyAlterAttributes class handles attributes that describe alternate key information for structured files. An AlternateKeyAlterAttribute object is returned from EnscribeFileAttributes.getAlternateKeyAttributes(). To change the alternate key attributes, use the set methods of this object, then pass this object as a parameter to the EnscribeFileAttributes.updateAlternateKeyAttributes() method. After you have made all the changes that you want to the attributes of a file, pass the EnscribeFileAttributes object to the EnscribeFile.alter() method.
The AlternateKeyDescriptor handles attributes that describe a specific alternate key. This class can be passed as a parameter when creating or altering a structured file. It is also returned when information is requested about a structured file.
EnscribeFileException exception class indicates that an error has occurred while trying an Enscribe File operation.
There are other classes which complement these functionalities.
CODE EXAMPLE
Let us look at a code example of creation of key sequenced file along with its alternate key file. Additionally let’s look at how to insert records and retrieve records.
"""
A sample program to create and write records to a
key sequenced file
"""
from py_enscribe.EnscribeFile import EnscribeFile
from py_enscribe.EnscribeFileException import EnscribeFileException
from py_enscribe.KeySequencedCreationAttributes import KeySequencedCreationAttributes
from py_enscribe.AlternateKeyDescriptor import AlternateKeyDescriptor
from py_enscribe.EnscribeOpenOptions import EnscribeOpenOptions
from py_enscribe.EnscribeKeyPositionOptions import EnscribeKeyPositionOptions
filename = "$oss.enscribe.sample"
altFileName = "$oss.enscribe.altfile"
record_list = [
"01BobbieChapman 700000Java Development Group 05",
"02Faye Brackett 780000Java Development Group 05",
"03Frank Butera 790000Java HotSpot Group 06",
"04David Schorow 1000000Java Guru Group 07",
"05Alan Joseph 700000Java HotSpot Group 06",
"06Emile Roth 800000Java HotSpot Group 06",
"07Bill Blosen 900000Java Management Group 09",
"08Ken Holt 1000000Java Management Group 09",
"09Dave Velasco 800000Java Pthreads Group 08",
"10Last Guy 50000Last Guy Group 99"]
# First create the file
try:
sample = EnscribeFile(fileName)
altfile = EnscribeFile(altFileName)
# If the file exists, purge it
try:
if(sample.isExistingFile()):
sample.purge();
except EnscribeFileException as pe:
print("Shouldn't get an exception on the purge because we check to see if it exists first "+pe.getMessage())
# If the alternate key file exists, purge it
try:
if(altfile.isExistingFile()):
altfile.purge();
except EnscribeFileException as pe:
print("Shouldn't get an exception on the purge because we check to see if it exists first "+pe.getMessage());
# Create the primary file
creationAttr = KeySequencedCreationAttributes();
# Set general file attributes
creationAttr.setRecordLength(56);
creationAttr.setBlockLength(4096);
creationAttr.setKeyLength(2);
creationAttr.setKeyOffset(0);
# Set alternate key attributes
ak = [0];
ak[0] = AlternateKeyDescriptor();
ak[0].setKeyOffset(54);
ak[0].setKeyLength(2);
ak[0].setKeySpecifier("DP");
ak[0].setAlternateKeyFileName(altFileName);
creationAttr.setAlternateKeyDescriptors(ak);
sample.create(creationAttr);
sample.createAltKeyFiles();
print(">>Created File")
# Obtain the attributes of the newly created file
print(">>Attributes of new file: ")
myAttr = sample.getFileInfo()
print(" File code = ",myAttr.getFileCode())
print(" File security = ",myAttr.getGuardianSecurityString());
print(" Primary keyoffset = ",myAttr.getKeyOffset());
print(" Maximum extents = ",myAttr.getMaximumExtents());
print(" Alternate Key Attributes: ");
altdesc = myAttr.getAlternateKeyAttributes();
# Only one, since that is how we created the file
print(" Number of alt keys = ",myAttr.getNumberOfAltkeys());
print(" Alternate key file name = ",altdesc[0].getAlternateKeyFileName());
print(" Key Specifier is ",altdesc[0].getKeySpecifierString());
# Open the file and write some data
options = EnscribeOpenOptions()
options.setAccess(EnscribeOpenOptions.READ_WRITE)
sample.open(options,EnscribeOpenOptions.SHARED);
print(">>Opened file. ");
for i in range(0, len(record_list)):
sample.write(record_list[i]);
print(">>Wrote data to the file");
print(" ");
# Going to position on alternate key value 06
print(">>Reading records with alternate key value of 06");
kpa = EnscribeKeyPositionOptions();
keySpec = altdesc[0].getKeySpecifierString();
kpa.setKeySpecifier(keySpec);
keyvalue = "06"
kpa.setKeyValue(keyvalue);
kpa.setPositioningMode(EnscribeKeyPositionOptions.POSITION_GENERIC);
kpa.setKeyLength(2);
kpa.setCompareLength(2);
sample.keyPosition(kpa);
maxRead = sample.getMaxReadCount();
for i in range(0,3):
buf, count = sample.read(maxRead);
print(" ",buf);
# Now read in the reverse direction
print("");
print(">>Reading same records in reverse direction");
kpa.setReadReversed(True,True);
sample.keyPosition(kpa);
record, count = sample.read(maxRead)
print(" ",record)
while count!= -1:
record, count = sample.read(maxRead)
print(" ",record);
sample.close();
except EnscribeFileException as ee:
print(ee.getMessage());
py_enscribe also helps in file_search using the EnscribeFileSearch module, internally it makes use of APIs like FILENAME_FINDSTART_, FILENAME_FINDNEXT_ and FILENAME_FINDFINISH_.
UNDER THE HOOD
The item codes and values for a GPC are interpreted as a list in the python layer. Any value item that has more than 2 bytes is to be split into successive values in intervals of two bytes in the value_list.
Let us take the item code 104, that is the names of alternate key files. Let’s say we have an alternate file with file name “altfile”, we need to map this to the value list as:
[“al”, “tf”, “il”, “e”]
The strings are internally converted to bytes strings before passing to cython .pyx file, and then strings are interpreted into integers using bit logic,
def bytes_to_int(bytes byte_string):
"""
this is used to convert a bytes object containing two or one
character into corresponding int. This is analogous to C types
cast of char * to short.
for ex: short key = (short)('DT'); /* key specifier */
"""
cdef int result = 0
cdef int length = len(byte_string)
cdef int i
if length == 2:
for i in range(length):
result = (result << 8) | byte_string[i]
elif length == 1:
result = byte_string[0] << 8
Once you have item-list and value-list ready you can pass it to a function defined in cython .pyx file. The desired function in cython .pyx file can import a GPC and call the respective GPC. The template goes as follows,
.pyx file contents
cdef extern from "cextdecs.h":
short FILE_CLOSE_(short filenum, short tape_disposition);
def file_close(filenum, tape_disposition=0):
""" closes open file with wait i/o """
error = FILE_CLOSE_(filenum, tape_disposition);
return error
In a .py file you can import the module and use file_close as a normal python function.
WRITING AND READING A PYTHON OBJECT FROM ENSCRIBE FILE
We can read and write a python class with all its methods to an Enscribe file. This will help with structured data storage and retrieval.
import base64
import pickle
from py_enscribe.EnscribeFile import EnscribeFile
from py_enscribe.EnscribeFileException import EnscribeFileException
from py_enscribe.KeySequencedCreationAttributes import KeySequencedCreationAttributes
from py_enscribe.AlternateKeyDescriptor import AlternateKeyDescriptor
from py_enscribe.EnscribeOpenOptions import EnscribeOpenOptions
from py_enscribe.EnscribeKeyPositionOptions import EnscribeKeyPositionOptions
fileName = "$oss.enscribe.samplc"
altFileName = "$oss.enscribe.altfilc"
class Employee:
def __init__(self, name, experience, company):
self.name = name
self.experience = experience
self.company = company
def __repr__(self):
return f"Employee(name={self.name}, company={self.company})"
def calc_salary(self, sal):
print("Employee salary is ", sal)
# Create an object
obj = Employee("John", 4, "HPE")
# Serialize (pack) into a byte string
byte_data = base64.b64encode(pickle.dumps(obj)).decode("utf-8")
print(f"Packed bytes: {byte_data}, data len is ", len(byte_data))
try:
sample = EnscribeFile(fileName)
altfile = EnscribeFile(altFileName)
# If the file exists,purge it
try:
if(sample.isExistingFile()):
sample.purge();
except EnscribeFileException as pe:
print("Shouldn't get an exception on the purge because we check to see if it exists first "+pe.getMessage())
# If the alternate key file exists, purge it
try:
if(altfile.isExistingFile()):
altfile.purge();
except EnscribeFileException as pe:
print("Shouldn't get an exception on the purge because we check to see if it exists first "+pe.getMessage());
# Create the primary file
creationAttr = KeySequencedCreationAttributes();
# Set general file attributes
creationAttr.setRecordLength(1024);
creationAttr.setBlockLength(4096);
creationAttr.setKeyLength(2);
creationAttr.setKeyOffset(0);
# Set alternate key attributes
ak = [0];
ak[0] = AlternateKeyDescriptor();
ak[0].setKeyOffset(54);
ak[0].setKeyLength(2);
ak[0].setKeySpecifier("DP");
ak[0].setAlternateKeyFileName(altFileName);
creationAttr.setAlternateKeyDescriptors(ak);
sample.create(creationAttr);
sample.createAltKeyFiles();
print(">>Created File")
options = EnscribeOpenOptions()
options.setAccess(EnscribeOpenOptions.READ_WRITE)
sample.open(options,EnscribeOpenOptions.SHARED);
print(">>Opened File ");
sample.write(byte_data)
print(">>Wrote data to the File");
print(" ");
print("Reading Data")
stored_str, _ = sample.read(1024)
restored_obj = pickle.loads(base64.b64decode(stored_str.encode("utf-8")))
print("Restored Object ",restored_obj)
print("Employee Name ", restored_obj.name)
print("Company Name ",restored_obj.company)
restored_obj.calc_salary(500)
sample.close()
except EnscribeFileException as ee:
print(ee.getMessage())
The above program outputs the following,
Packed bytes: gASVTgAAAAAAAACMCF9fbWFpbl9flIwIRW1wbG95ZWWUk5QpgZR9lCiMBG5hbWWUjARKb2hulIwKZXhwZXJpZW5jZZRLBIwHY29tcGFueZSMA0hQRZR1Yi4=, data len is 120 >>Created File >>Opened File >>Wrote data to the File Reading Data Restored Object Employee(name=John, company=HPE) Employee Name John Company Name HPE Employee salary is 500
ADVANTAGES
It is easier to call GPCs using cython. Creating C extensions leads to faster execution time rather than having to load the desired dll using ctypes. Remembering the item descriptions is easier than mapping item codes to values for a GPC. The py_enscribe takes care of corner cases and issues that can arise due to an Enscribe operation using the EnscribeFileException module.
Be the first to comment